-- Logs begin at Mon 2022-06-06 09:56:22 UTC, end at Mon 2022-06-06 10:59:03 UTC. -- Jun 06 10:23:12.156442 dv-cinder-844822-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 06 10:23:12.165856 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 06 10:23:12.167861 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jun 6 10:23:12 2022] *** Jun 06 10:23:12.167861 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 06 10:23:12.167861 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 06 10:23:12.167861 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: nodename: dv-cinder-844822-1-iscsi Jun 06 10:23:12.167861 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: machine: x86_64 Jun 06 10:23:12.167861 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: clock source: unix Jun 06 10:23:12.167861 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: pcre jit disabled Jun 06 10:23:12.167861 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: detected number of CPU cores: 4 Jun 06 10:23:12.167861 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: current working directory: / Jun 06 10:23:12.167861 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: detected binary path: /usr/bin/uwsgi-core Jun 06 10:23:12.168401 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: your processes number limit is 47680 Jun 06 10:23:12.168401 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: your memory page size is 4096 bytes Jun 06 10:23:12.168401 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: detected max file descriptor number: 2048 Jun 06 10:23:12.168401 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: lock engine: pthread robust mutexes Jun 06 10:23:12.168401 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: thunder lock: enabled Jun 06 10:23:12.168401 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 06 10:23:12.168401 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 06 10:23:12.212632 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: Python main interpreter initialized at 0x55cebe02c9c0 Jun 06 10:23:12.212632 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: python threads support enabled Jun 06 10:23:12.212632 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: your server socket listen backlog is limited to 100 connections Jun 06 10:23:12.212632 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: your mercy for graceful operations on workers is 90 seconds Jun 06 10:23:12.212955 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: mapped 403077 bytes (393 KB) for 2 cores Jun 06 10:23:12.213052 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: *** Operational MODE: preforking *** Jun 06 10:23:12.213327 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: *** uWSGI is running in multiple interpreter mode *** Jun 06 10:23:12.213327 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: spawned uWSGI master process (pid: 80230) Jun 06 10:23:12.213519 dv-cinder-844822-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 06 10:23:12.214369 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: spawned uWSGI worker 1 (pid: 80240, cores: 1) Jun 06 10:23:12.214369 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: spawned uWSGI worker 2 (pid: 80241, cores: 1) Jun 06 10:23:12.215826 dv-cinder-844822-1-iscsi devstack@keystone.service[80230]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 06 10:23:17.912952 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 06 10:23:17.912952 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: from cryptography.utils import int_from_bytes Jun 06 10:23:17.913746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 06 10:23:17.913746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: from cryptography.utils import int_from_bytes Jun 06 10:23:17.913975 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 06 10:23:17.913975 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: from cryptography.utils import int_from_bytes Jun 06 10:23:17.914465 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 06 10:23:17.914465 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: from cryptography.utils import int_from_bytes Jun 06 10:23:17.963417 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 06 10:23:17.963599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 06 10:23:17.963804 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] command line args: [] {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 06 10:23:17.963910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 06 10:23:17.964055 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 06 10:23:17.964146 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 06 10:23:17.964225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ================================================================================ {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 06 10:23:17.964300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] command line args: [] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 06 10:23:17.964480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] admin_token = **** {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.964559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 06 10:23:17.964679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ================================================================================ {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 06 10:23:17.964768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] config_dir = [] {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.964900 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.964977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] admin_token = **** {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.965084 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] config_source = [] {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.965167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] config_dir = [] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.965263 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] debug = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.965347 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.965460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.965772 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] config_source = [] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.965772 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] debug = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.965965 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.966066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.966198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.966293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.966293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.966469 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.966568 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.966672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.966672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] list_limit = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.966851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.966950 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] log_config_append = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.967052 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] list_limit = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.967151 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.967248 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] log_config_append = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.967340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] log_dir = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.967436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.967540 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] log_file = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.967642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] log_dir = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.967642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] log_file = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.967821 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.967947 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.968022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.968140 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.968223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.968299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.968411 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.968486 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.968605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.968686 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.968807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.968807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.968992 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.969065 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.969178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.969281 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.969359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.969514 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.969628 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.969744 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.969818 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.969970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.970047 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.970199 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.970287 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.970442 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.970525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.970683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.970762 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.970951 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.971187 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.971374 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] publish_errors = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.971459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.971621 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] publish_errors = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.971767 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.971874 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.971992 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.972072 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.972141 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.972214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.972282 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.972471 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.972558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.972681 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.972763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] standard_threads = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.972912 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.972995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] standard_threads = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.973138 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.973218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.973333 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.973411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.973489 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] use_journal = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.973559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.973633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] use_json = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.973707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] use_journal = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.973809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] use_stderr = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.973884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] use_json = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.973986 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] use_syslog = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.974069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] use_stderr = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.974176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] use_syslog = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.974249 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.974403 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 10:23:17.974484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.974655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.974733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.974888 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.974969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.975142 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.975223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.975378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.975464 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.975614 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.975689 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.975902 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.976125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.976125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.976540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.976540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.976667 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.976736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.976804 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.976879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.976950 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.977092 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.977177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.977337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.977425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.977574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.977741 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.977824 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.977982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.978058 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.978230 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.978314 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.978475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.978550 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.978709 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.978784 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.978961 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.979049 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.979207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.979287 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.979445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.979520 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.979679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.979767 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.979955 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.980033 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.980214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.980289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.980448 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.980535 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.980681 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.980762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.980910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.980985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.981137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.981220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.981366 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.981448 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.981599 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.981675 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.981841 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.981918 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.982072 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.982154 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.982296 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.982370 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.982523 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.982601 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.982767 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.982842 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.983010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.983086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.983241 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.983316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.983476 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.983551 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.983700 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.983774 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.983957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.984035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.984185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.984259 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.984412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.984493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.984638 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.984715 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.984863 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.984943 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.985089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.985164 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.985322 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.985399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.985558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.985637 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.985784 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.985887 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.986015 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.986121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.986242 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.986343 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.986473 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.986582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.986689 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.986765 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.986929 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.987054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.987155 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.987230 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.987389 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.987469 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.987622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.987743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.987880 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.987957 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.988111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.988190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.988342 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.988418 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.988567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.988646 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.988806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.988888 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.989036 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.989120 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.989268 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.989346 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.989496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.989587 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.989739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.989821 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.989981 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.990126 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.990200 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.990278 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.990424 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.990506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.990656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.990728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.990920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.990996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.991151 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.991227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.991388 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.991464 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.992536 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.992631 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.992706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.992772 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.992846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.992916 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.993028 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.993114 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.993267 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.993342 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.993499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.993575 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.993733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.993880 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.993954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.994023 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.994191 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.994273 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.994421 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.994505 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.994653 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.994728 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.994882 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.994957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.995113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.995196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.995369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.995446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.995605 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.995679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.995861 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.995952 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.996097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.996176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.996331 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.996413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.996562 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.996644 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.996789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.996876 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.997017 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.997092 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.997246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.997327 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.997471 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.997553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.997697 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.997773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.997934 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.998008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.998160 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.998242 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.998398 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.998477 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.998630 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.998706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.998874 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.998959 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.999107 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.999183 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.999357 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.999433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.999590 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] auth.external = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.999672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.999838 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:17.999918 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.000073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] auth.external = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.000246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.000334 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.000488 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.000578 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.000725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] auth.password = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.000802 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.000959 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] auth.token = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.001036 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] auth.password = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.001205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.001288 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] auth.token = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.001448 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.001524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.001681 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.002023 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.002023 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.002172 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.002172 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.002301 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.002372 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.002441 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.002599 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.002687 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.002827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.002904 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] credential.caching = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.003062 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] credential.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.003140 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.003305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.003389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.003552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.003629 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.003813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.003913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.004039 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.004129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.004278 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.004355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.004511 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.004587 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.004768 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.004852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.005010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.005086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.005243 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.005319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.005492 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.005577 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.005768 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.005845 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.006010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.006088 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.006262 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.006346 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.006500 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.006576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.006747 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] federation.caching = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.006823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.006979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] federation.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.007062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.007852 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.008311 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.008388 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.008459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.008459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.008680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.008757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.008828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.008899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.008968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.009038 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.009117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.009226 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.009310 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.009484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.009567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.009724 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.009809 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.009965 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.010047 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.010202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] identity.caching = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.010277 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.010432 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.010516 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.010678 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.010755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.010912 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.010994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.011143 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.011220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.011374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.011451 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.011608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.011890 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.012136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.012388 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.012621 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.012853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.013085 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.013167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.013329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.013407 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.013574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.013657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.013822 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.013898 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.014077 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.014170 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.014321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.014399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.014560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.014636 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.014800 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.014879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.015038 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.015113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.015269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.015345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.015504 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.015583 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.015734 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.015825 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.015995 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.016072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.016237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.016317 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.016478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.016556 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.016726 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.016803 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.016958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.017036 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.017198 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.017273 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.017428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.017505 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.017663 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.017742 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.017904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.017980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.018143 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.018219 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.018371 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.018447 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.018607 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.018683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.018836 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.018912 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.019073 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.019152 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.019305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.019380 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.019543 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.019620 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.019828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.019910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.020029 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.020106 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.020265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.020342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.020493 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.020571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.020726 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.020802 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.020957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.021033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.021189 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.021266 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.021421 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.021497 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.021653 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.021730 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.021887 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.021964 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.022127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.022202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.022353 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.022429 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.user = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.022582 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.022660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.022896 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.023076 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.023076 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.023524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.023599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.023952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.024030 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.024101 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.024172 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.024255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.024328 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.024398 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.024527 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.024605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.024763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.024849 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.025000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.025077 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.025239 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.025316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.025470 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.025555 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.025717 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.025794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.025961 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.026037 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.026190 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.026279 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.026423 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.026501 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.026657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.026738 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.026883 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.026967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.027112 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.027188 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.027353 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.027429 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.027589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.027674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.027850 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.027939 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.028086 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.028162 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.028325 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.028412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.028552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.028630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.028791 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.028866 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.029021 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.029104 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.029253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.029329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.029485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.029561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.029715 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.029800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.029951 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.030026 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.030187 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.030266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.030418 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.030501 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.030664 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.030741 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.030889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.030964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.031121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.031204 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.031351 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.031426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] resource.caching = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.031584 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.031662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.031838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.031997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.032073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.032143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.032309 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.032387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.032537 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.032622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.032770 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.032847 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.033008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] resource.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.033084 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.033242 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.033324 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.033479 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.033555 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.033706 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.033787 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] role.caching = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.033951 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] role.driver = sql {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.034035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.034184 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.034261 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.034414 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.034489 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.034653 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.034738 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.034886 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.034963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.035118 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.035195 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.035349 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] role.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.035433 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.035590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] role.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.035667 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.035832 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.035913 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.036078 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.036167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.036322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.036398 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.036564 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.036639 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.036808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.036887 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.037054 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.037129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.037293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.037368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.037535 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.037686 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.037769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.037923 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.038000 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.038176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.038261 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.038481 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.039029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.039117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.039190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.039261 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.039342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.039342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.039467 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.039538 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.039608 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.039742 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.039860 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.040008 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.040086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.040242 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.040326 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.040476 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.040565 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.040708 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.040785 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.040954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.041031 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.041205 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.041289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.041441 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.041517 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.041667 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.041744 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.041894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.041972 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.042120 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.042196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.042354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] token.caching = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.042429 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.042589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.042668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.042831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.042912 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.043065 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.043140 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.043293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.043369 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.043519 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.043595 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.043751 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] token.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.043846 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.044005 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.044085 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.044248 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.044326 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.044479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.044556 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.044716 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.044794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.044954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.045030 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.045184 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.045259 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.045425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.045511 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.045666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.045743 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.045902 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.045977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.046131 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.046207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.046365 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.046443 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.046596 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.046672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] database.connection = **** {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.046840 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.046927 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.047083 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.047161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.047308 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.047384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.047546 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.047622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.047861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.047941 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.048103 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.048179 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.connection = **** {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.048340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.048427 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.048578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.048666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.048808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.048884 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.049044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.049120 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.049275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.049359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.049507 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.049584 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.049738 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.049815 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.049969 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.050053 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.050206 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.050283 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.050434 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.050519 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.050670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.050821 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.050897 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.050961 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.051063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 06 10:23:18.051140 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.051346 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.051583 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.051839 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.052062 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.052293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 10:23:18.052458 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 06 10:23:18.081837 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.082006 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.082087 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.082161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.082276 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.082358 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.091028 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.091338 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.119300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.119519 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.119957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.120193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.172828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.173012 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.221273 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.221503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.222256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.222495 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.287614 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.287940 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.288126 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.288346 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.288583 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.288797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.289078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.289316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.289541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.289760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.289975 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.290190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.312082 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.312305 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.312428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.312510 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.312585 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.312732 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.312819 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.312953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.313036 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.313170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.313252 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.313376 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.313456 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.313586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.324988 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.325090 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.338404 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.338593 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.338709 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.338788 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.338862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.338924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.339072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.339164 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.339281 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.339362 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.339488 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.339570 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.339696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.339806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 10:23:18.356976 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.357183 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 06 10:23:18.357513 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.357626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.357722 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 06 10:23:18.357840 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 06 10:23:18.357957 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.358174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.358262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.358376 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 06 10:23:18.358464 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.358644 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.358724 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.358904 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.359159 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.359410 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.369903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.370356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.370784 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.371238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.372156 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.372618 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.373059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.373547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.381757 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.381924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.384959 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.384959 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.385100 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.385255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.385532 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.385784 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.385904 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.386044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.386185 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.386309 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.386466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.386570 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.386744 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.386831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.387021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.387109 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.387278 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.387544 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.387831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.388098 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.388367 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.388632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.414131 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.415714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.416011 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.416150 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.417763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.418042 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.421345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.421619 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.421889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.423454 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.423694 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.424003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.429219 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.429493 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.431360 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.431644 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.433914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.434194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.434467 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.434739 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.435004 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.435266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.435525 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.435921 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.436150 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.436437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.436718 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.436999 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.437271 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.437540 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.437804 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.438178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.455368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.455634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.457573 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.457851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.460239 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.462979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.464533 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.464820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.466742 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.467034 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.469793 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.472065 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.473382 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.475652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.477150 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.477437 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.477712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.477988 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.478267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.478539 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.479326 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.479622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.479935 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.480219 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.480503 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.480786 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.497563 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.497837 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.498094 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.498349 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.499939 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.500221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.500492 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.500750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.506196 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.509172 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.510000 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.510290 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.511000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.511274 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.513382 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.513653 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.513922 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.514193 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.514360 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.514454 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.514642 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.514915 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.515188 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.515470 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.528624 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.528902 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.529179 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.529456 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.529738 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.530010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.530288 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.530849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.531128 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.531412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.531693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.532006 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.532281 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.532553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.548080 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.550351 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.551297 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.553614 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.554801 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.556415 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.557335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.558037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.558308 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.558580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.558935 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.560617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.560895 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.561170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.565859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.567450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.568437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.570680 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.571356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.571638 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.571946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.572217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.574044 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.574332 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.574614 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.574894 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.581245 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.581526 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.581796 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.583956 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.584239 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.584515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.589822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.590097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.590368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.590636 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.590900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.591162 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.591437 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.591716 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.592007 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.592269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.592530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.592609 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 10:23:18.592788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.592872 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.593053 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.593142 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.593312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.593396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.593575 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.593860 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.594132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.594401 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.594674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.594946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.595212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.595478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.595743 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.596032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 10:23:18.632193 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cebe02c9c0 pid: 80240 (default app) Jun 06 10:23:18.635015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cebe02c9c0 pid: 80241 (default app) Jun 06 10:23:18.635599 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfdc55-07b4-4a77-90ff-f75e5eccd2f4 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:18.635917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfdc55-07b4-4a77-90ff-f75e5eccd2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:18.636191 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfdc55-07b4-4a77-90ff-f75e5eccd2f4 None None] PATH_INFO: `/v3` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:18.640368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Mon Jun 6 10:23:18 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:26.655692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37f9cd-08f0-42e7-9bac-4594c9a10c73 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:26.656260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37f9cd-08f0-42e7-9bac-4594c9a10c73 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:26.656260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37f9cd-08f0-42e7-9bac-4594c9a10c73 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:26.660442 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:23:26.666330 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91f4af-59c3-472d-8803-3fd226747ddb None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:26.666745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91f4af-59c3-472d-8803-3fd226747ddb None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:26.666913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91f4af-59c3-472d-8803-3fd226747ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:26.733783 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: INFO dbcounter [None req-2d91f4af-59c3-472d-8803-3fd226747ddb None None] Registered counter for database keystone Jun 06 10:23:26.755189 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG oslo_db.sqlalchemy.engines [None req-2d91f4af-59c3-472d-8803-3fd226747ddb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80240) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 06 10:23:26.761368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writer thread running {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 06 10:23:27.039367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG passlib.handlers.bcrypt [None req-2d91f4af-59c3-472d-8803-3fd226747ddb None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80240) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 06 10:23:27.039815 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG passlib.handlers.bcrypt [None req-2d91f4af-59c3-472d-8803-3fd226747ddb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80240) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 06 10:23:27.095249 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2d91f4af-59c3-472d-8803-3fd226747ddb None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:27.096864 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2d91f4af-59c3-472d-8803-3fd226747ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:27.217366 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 552 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 06 10:23:27.415377 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37f9cd-08f0-42e7-9bac-4594c9a10c73 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:27.483577 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: INFO dbcounter [None req-8c37f9cd-08f0-42e7-9bac-4594c9a10c73 None None] Registered counter for database keystone Jun 06 10:23:27.502767 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG oslo_db.sqlalchemy.engines [None req-8c37f9cd-08f0-42e7-9bac-4594c9a10c73 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80241) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 06 10:23:27.508943 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writer thread running {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 06 10:23:27.731426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37f9cd-08f0-42e7-9bac-4594c9a10c73 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:27.735620 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85661de1-186a-43a9-98af-6ee193d9f6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:27.736301 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85661de1-186a-43a9-98af-6ee193d9f6db admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:27.736584 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85661de1-186a-43a9-98af-6ee193d9f6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:27.736857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85661de1-186a-43a9-98af-6ee193d9f6db admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:27.745074 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85661de1-186a-43a9-98af-6ee193d9f6db admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:27.796808 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: WARNING keystone.common.rbac_enforcer.enforcer [None req-85661de1-186a-43a9-98af-6ee193d9f6db admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 06 10:23:27.797147 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85661de1-186a-43a9-98af-6ee193d9f6db admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:27.806509 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Mon Jun 6 10:23:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 10:23:27.812543 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91f4af-59c3-472d-8803-3fd226747ddb None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:27.827001 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91f4af-59c3-472d-8803-3fd226747ddb None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:27.831415 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7600d0b-3471-46b6-b6ad-6817e0056ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:27.832587 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7600d0b-3471-46b6-b6ad-6817e0056ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:27.832874 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7600d0b-3471-46b6-b6ad-6817e0056ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:27.833151 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7600d0b-3471-46b6-b6ad-6817e0056ce5 admin admin] PATH_INFO: `/v3/projects/863195bccbf14e04a9a61a90f52ba9f1` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:27.834648 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7600d0b-3471-46b6-b6ad-6817e0056ce5 admin admin] RBAC: Authorizing `identity:get_project(project_id=863195bccbf14e04a9a61a90f52ba9f1)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:27.885150 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d7600d0b-3471-46b6-b6ad-6817e0056ce5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 06 10:23:27.885491 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7600d0b-3471-46b6-b6ad-6817e0056ce5 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:27.891266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Mon Jun 6 10:23:27 2022] GET /identity/v3/projects/863195bccbf14e04a9a61a90f52ba9f1 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:30.117775 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c03482e-18df-4f4a-b27e-dc0fa4e589b6 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:30.118322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c03482e-18df-4f4a-b27e-dc0fa4e589b6 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:30.118322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c03482e-18df-4f4a-b27e-dc0fa4e589b6 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:30.122440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:23:30.127860 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69a929-4d17-4a79-8bea-354816a95db0 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:30.128272 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69a929-4d17-4a79-8bea-354816a95db0 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:30.128416 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69a929-4d17-4a79-8bea-354816a95db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:30.147969 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-fc69a929-4d17-4a79-8bea-354816a95db0 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:30.149864 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fc69a929-4d17-4a79-8bea-354816a95db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:30.157771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 06 10:23:30.180360 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c03482e-18df-4f4a-b27e-dc0fa4e589b6 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:30.192378 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c03482e-18df-4f4a-b27e-dc0fa4e589b6 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:30.197254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353daf4-0476-4187-9438-75a4a36f3321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:30.197882 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353daf4-0476-4187-9438-75a4a36f3321 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:30.198166 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353daf4-0476-4187-9438-75a4a36f3321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:30.198441 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353daf4-0476-4187-9438-75a4a36f3321 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:30.206007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a353daf4-0476-4187-9438-75a4a36f3321 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:30.207758 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a353daf4-0476-4187-9438-75a4a36f3321 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:30.216469 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Mon Jun 6 10:23:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 10:23:30.221751 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69a929-4d17-4a79-8bea-354816a95db0 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:30.231353 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69a929-4d17-4a79-8bea-354816a95db0 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:30.235706 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfcca98-7f8c-40f7-a098-293d4a2a2d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:30.236588 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfcca98-7f8c-40f7-a098-293d4a2a2d94 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:30.236875 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfcca98-7f8c-40f7-a098-293d4a2a2d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:30.237154 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfcca98-7f8c-40f7-a098-293d4a2a2d94 admin admin] PATH_INFO: `/v3/users/dc953d765b8048419427028bb5821273` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:30.238531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfcca98-7f8c-40f7-a098-293d4a2a2d94 admin admin] RBAC: Authorizing `identity:get_user(user_id=dc953d765b8048419427028bb5821273)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:30.240397 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfcca98-7f8c-40f7-a098-293d4a2a2d94 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:30.242711 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Mon Jun 6 10:23:30 2022] GET /identity/v3/users/dc953d765b8048419427028bb5821273 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:32.561683 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca18161-ee94-4b40-ae5d-99a30b274c71 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:32.562216 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef5dc9-f2da-4535-b4ef-5d6a1fd1f62d None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:32.562586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca18161-ee94-4b40-ae5d-99a30b274c71 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:32.562677 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef5dc9-f2da-4535-b4ef-5d6a1fd1f62d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:32.562757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca18161-ee94-4b40-ae5d-99a30b274c71 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:32.562832 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef5dc9-f2da-4535-b4ef-5d6a1fd1f62d None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:32.563575 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:23:32.564050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:23:32.568433 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ff629-ba39-4731-9efe-64b4508ba4b0 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:32.568595 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229eeab5-28cb-4715-85cb-73b74f3d9798 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:32.568720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ff629-ba39-4731-9efe-64b4508ba4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:32.568884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229eeab5-28cb-4715-85cb-73b74f3d9798 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:32.569015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ff629-ba39-4731-9efe-64b4508ba4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:32.569168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229eeab5-28cb-4715-85cb-73b74f3d9798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:32.585811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-229eeab5-28cb-4715-85cb-73b74f3d9798 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:32.586996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-229eeab5-28cb-4715-85cb-73b74f3d9798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:32.607917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG passlib.handlers.bcrypt [None req-4e0ff629-ba39-4731-9efe-64b4508ba4b0 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80241) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 06 10:23:32.608337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG passlib.handlers.bcrypt [None req-4e0ff629-ba39-4731-9efe-64b4508ba4b0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80241) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 06 10:23:32.654868 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-4e0ff629-ba39-4731-9efe-64b4508ba4b0 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:32.656397 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-4e0ff629-ba39-4731-9efe-64b4508ba4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:32.666324 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 7/12] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 10:23:32.688859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229eeab5-28cb-4715-85cb-73b74f3d9798 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:32.711860 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:32.755649 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 6/13] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 10:23:32.779096 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ff629-ba39-4731-9efe-64b4508ba4b0 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:32.798603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229eeab5-28cb-4715-85cb-73b74f3d9798 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:32.831739 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2721d0-959d-49b6-8008-94d41403bcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:32.832412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2721d0-959d-49b6-8008-94d41403bcff None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:32.832703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2721d0-959d-49b6-8008-94d41403bcff None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:32.832977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2721d0-959d-49b6-8008-94d41403bcff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:32.878326 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ff629-ba39-4731-9efe-64b4508ba4b0 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:32.880609 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:32.901163 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2721d0-959d-49b6-8008-94d41403bcff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:32.902762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2721d0-959d-49b6-8008-94d41403bcff None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:32.917250 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef17f8e-c4d2-4132-bc87-3b8e87699ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:32.918227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef17f8e-c4d2-4132-bc87-3b8e87699ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:32.918516 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef17f8e-c4d2-4132-bc87-3b8e87699ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:32.918794 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef17f8e-c4d2-4132-bc87-3b8e87699ed7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:32.925961 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:32.931004 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef17f8e-c4d2-4132-bc87-3b8e87699ed7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:23:32.942202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-2ef17f8e-c4d2-4132-bc87-3b8e87699ed7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:23:32.942202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:23:32.942202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:23:32.942202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:23:32.942202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:23:32.942202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:23:32.942202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:23:32.942202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:23:32.942202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:23:32.942202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:23:32.942202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:23:32.942202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:23:32.942202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:23:32.942202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:23:32.942202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:23:32.942202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:23:32.943214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:23:32.943214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:23:32.943214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:23:32.943214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:23:32.943214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:23:32.943214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:23:32.943214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:23:32.943214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:23:32.943214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:23:32.943214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:23:32.943214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:23:32.943214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:23:32.943214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:23:32.943214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:23:32.943214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:23:32.943214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:23:32.944109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:23:32.944109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:23:32.944109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:23:32.944109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:23:32.944109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:23:32.944109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:23:32.944109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:23:32.944109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:23:32.944109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:23:32.944109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:23:32.944109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:23:32.944109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:23:32.944109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:23:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:23:32.949623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef17f8e-c4d2-4132-bc87-3b8e87699ed7 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:33.005601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 8/15] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:23:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 06 10:23:33.011124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2721d0-959d-49b6-8008-94d41403bcff None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:33.050165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef17f8e-c4d2-4132-bc87-3b8e87699ed7 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:33.082877 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2df86e-70e2-498f-8dd4-0212c27752b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:33.083534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2df86e-70e2-498f-8dd4-0212c27752b0 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:33.083839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2df86e-70e2-498f-8dd4-0212c27752b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:33.084118 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2df86e-70e2-498f-8dd4-0212c27752b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:33.085407 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2df86e-70e2-498f-8dd4-0212c27752b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:33.085798 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2df86e-70e2-498f-8dd4-0212c27752b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:33.087237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2df86e-70e2-498f-8dd4-0212c27752b0 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:33.102384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 8/16] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:23:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:23:33.106546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2df86e-70e2-498f-8dd4-0212c27752b0 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:33.110986 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2721d0-959d-49b6-8008-94d41403bcff None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:33.132044 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:33.134047 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:33.146294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d68b8d-8fa6-4434-83f3-f4b325a6abb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:33.147105 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d68b8d-8fa6-4434-83f3-f4b325a6abb0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:33.147930 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d68b8d-8fa6-4434-83f3-f4b325a6abb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:33.148208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d68b8d-8fa6-4434-83f3-f4b325a6abb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:33.159339 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d68b8d-8fa6-4434-83f3-f4b325a6abb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:33.160909 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d68b8d-8fa6-4434-83f3-f4b325a6abb0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:33.182010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-60d68b8d-8fa6-4434-83f3-f4b325a6abb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:23:33.182010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:23:33.182010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:23:33.182010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:23:33.182010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:23:33.182010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:23:33.182010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:23:33.182010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:23:33.182010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:23:33.182010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:23:33.182010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:23:33.182010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:23:33.182010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:23:33.182010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:23:33.182010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:23:33.182857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:23:33.182857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:23:33.182857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:23:33.182857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:23:33.182857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:23:33.182857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:23:33.182857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:23:33.182857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:23:33.182857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:23:33.182857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:23:33.182857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:23:33.182857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:23:33.182857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:23:33.182857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:23:33.182857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:23:33.182857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:23:33.182857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:23:33.183651 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:23:33.183651 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:23:33.183651 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:23:33.183651 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:23:33.183651 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 9/17] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:23:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:23:33.189036 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d68b8d-8fa6-4434-83f3-f4b325a6abb0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:33.206433 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2df86e-70e2-498f-8dd4-0212c27752b0 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:33.241236 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bcee89-0200-469c-a207-ecaae87aec43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:33.242241 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bcee89-0200-469c-a207-ecaae87aec43 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:33.242529 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bcee89-0200-469c-a207-ecaae87aec43 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:33.242809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bcee89-0200-469c-a207-ecaae87aec43 None admin] PATH_INFO: `/v3/users/dc953d765b8048419427028bb5821273` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:33.244399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bcee89-0200-469c-a207-ecaae87aec43 None admin] RBAC: Authorizing `identity:get_user(user_id=dc953d765b8048419427028bb5821273)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:33.245883 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bcee89-0200-469c-a207-ecaae87aec43 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:33.247613 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 9/18] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:23:33 2022] GET /identity/v3/users/dc953d765b8048419427028bb5821273 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:23:33.252800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bcee89-0200-469c-a207-ecaae87aec43 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:33.287843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d68b8d-8fa6-4434-83f3-f4b325a6abb0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:33.320523 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d754b-2195-4d0c-bcfc-2a35b6d8b663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:33.321183 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d754b-2195-4d0c-bcfc-2a35b6d8b663 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:33.321468 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d754b-2195-4d0c-bcfc-2a35b6d8b663 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:33.321747 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d754b-2195-4d0c-bcfc-2a35b6d8b663 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:33.322466 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d754b-2195-4d0c-bcfc-2a35b6d8b663 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:33.322773 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d754b-2195-4d0c-bcfc-2a35b6d8b663 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:33.324182 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d754b-2195-4d0c-bcfc-2a35b6d8b663 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:33.341458 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 10/19] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:23:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:33.344766 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:33.352272 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bcee89-0200-469c-a207-ecaae87aec43 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:33.392037 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2bfef-7032-4d6a-afba-609515084401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:33.392842 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2bfef-7032-4d6a-afba-609515084401 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:33.393122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2bfef-7032-4d6a-afba-609515084401 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:33.393395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2bfef-7032-4d6a-afba-609515084401 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:33.394587 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb2bfef-7032-4d6a-afba-609515084401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:33.396074 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb2bfef-7032-4d6a-afba-609515084401 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:33.398330 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 10/20] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:23:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:23:33.404646 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d754b-2195-4d0c-bcfc-2a35b6d8b663 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:33.440245 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:33.510915 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d754b-2195-4d0c-bcfc-2a35b6d8b663 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:33.546197 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1977d990-e00b-4a15-829a-308aea925708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:33.546852 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977d990-e00b-4a15-829a-308aea925708 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:33.547133 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977d990-e00b-4a15-829a-308aea925708 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:33.547408 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977d990-e00b-4a15-829a-308aea925708 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:33.548274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1977d990-e00b-4a15-829a-308aea925708 None admin] RBAC: Adding query filter params (role.id=0f929204eb7b4ea8859338ed2d3ff75b, scope.domain.id=default, user.id=dc953d765b8048419427028bb5821273) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:33.548581 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1977d990-e00b-4a15-829a-308aea925708 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:33.550031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1977d990-e00b-4a15-829a-308aea925708 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:33.557086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 11/21] 10.222.0.42 () {56 vars in 1322 bytes} [Mon Jun 6 10:23:33 2022] GET /identity/v3/role_assignments?user.id=dc953d765b8048419427028bb5821273&scope.domain.id=default&role.id=0f929204eb7b4ea8859338ed2d3ff75b => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:36.009099 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5d3aa-099c-45bd-a955-c17d3d1b639c None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:36.010139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5d3aa-099c-45bd-a955-c17d3d1b639c None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:36.010628 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5d3aa-099c-45bd-a955-c17d3d1b639c None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:36.012211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 11/22] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:23:36.017935 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e7c73-3ba1-42a1-baa7-ab6d8cc9b141 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:36.018766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e7c73-3ba1-42a1-baa7-ab6d8cc9b141 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:36.019253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e7c73-3ba1-42a1-baa7-ab6d8cc9b141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:36.044831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-8a3e7c73-3ba1-42a1-baa7-ab6d8cc9b141 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:36.046372 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-8a3e7c73-3ba1-42a1-baa7-ab6d8cc9b141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:36.140976 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 12/23] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:23:36.181149 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5d3aa-099c-45bd-a955-c17d3d1b639c None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:36.198920 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2ede0-3a5f-4f3b-962b-9009c09b9788 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:36.201887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2ede0-3a5f-4f3b-962b-9009c09b9788 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:36.202382 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2ede0-3a5f-4f3b-962b-9009c09b9788 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:36.204022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 13/24] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:23:36.210743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750532b-58c4-4c1c-8fd5-11909cd4126c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:36.211462 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750532b-58c4-4c1c-8fd5-11909cd4126c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:36.211995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750532b-58c4-4c1c-8fd5-11909cd4126c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:36.252675 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-5750532b-58c4-4c1c-8fd5-11909cd4126c None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:36.255591 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5750532b-58c4-4c1c-8fd5-11909cd4126c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:36.321599 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:36.415165 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 14/25] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:23:36.449465 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5d3aa-099c-45bd-a955-c17d3d1b639c None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:36.469450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5750532b-58c4-4c1c-8fd5-11909cd4126c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:36.517474 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c74d52-246f-404f-997b-8d3520870cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:36.518280 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c74d52-246f-404f-997b-8d3520870cda None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:36.518564 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c74d52-246f-404f-997b-8d3520870cda None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:36.518842 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c74d52-246f-404f-997b-8d3520870cda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:36.555094 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c74d52-246f-404f-997b-8d3520870cda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:36.556780 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c74d52-246f-404f-997b-8d3520870cda None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:36.598703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-87c74d52-246f-404f-997b-8d3520870cda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:23:36.598703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:23:36.598703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:23:36.598703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:23:36.598703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:23:36.598703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:23:36.598703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:23:36.598703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:23:36.598703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:23:36.598703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:23:36.598703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:23:36.598703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:23:36.598703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:23:36.598703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:23:36.598703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:23:36.599554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:23:36.599554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:23:36.599554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:23:36.599554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:23:36.599554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:23:36.599554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:23:36.599554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:23:36.599554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:23:36.599554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:23:36.599554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:23:36.599554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:23:36.599554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:23:36.599554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:23:36.599554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:23:36.599554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:23:36.599554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:23:36.599554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:23:36.601741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:23:36.601741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:23:36.601741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:23:36.601741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:23:36.601741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 12/26] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:23:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:23:36.609271 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d332e-44af-404b-ae35-c000c5f5c335 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:36.609752 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d332e-44af-404b-ae35-c000c5f5c335 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:36.610199 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d332e-44af-404b-ae35-c000c5f5c335 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:36.611677 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 13/27] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:23:36.612518 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d332e-44af-404b-ae35-c000c5f5c335 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:36.645770 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:36.656455 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5750532b-58c4-4c1c-8fd5-11909cd4126c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:36.712011 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0aacf3-b88a-4ac2-9861-db6202ec3bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:36.713467 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0aacf3-b88a-4ac2-9861-db6202ec3bf1 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:36.713892 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0aacf3-b88a-4ac2-9861-db6202ec3bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:36.714323 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0aacf3-b88a-4ac2-9861-db6202ec3bf1 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:36.715233 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0aacf3-b88a-4ac2-9861-db6202ec3bf1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:36.717413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0aacf3-b88a-4ac2-9861-db6202ec3bf1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:36.773157 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d332e-44af-404b-ae35-c000c5f5c335 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:36.810914 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22214f8-1e38-4ec1-8d06-cc5ae99d9d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:36.811999 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22214f8-1e38-4ec1-8d06-cc5ae99d9d88 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:36.812486 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22214f8-1e38-4ec1-8d06-cc5ae99d9d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:36.812984 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22214f8-1e38-4ec1-8d06-cc5ae99d9d88 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:36.813916 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22214f8-1e38-4ec1-8d06-cc5ae99d9d88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:36.814419 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22214f8-1e38-4ec1-8d06-cc5ae99d9d88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:36.816146 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22214f8-1e38-4ec1-8d06-cc5ae99d9d88 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:36.837288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 14/28] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:36.839124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735a270-4eb1-4b2b-bd58-f77200ceda0f None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:36.839654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735a270-4eb1-4b2b-bd58-f77200ceda0f None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:36.840165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735a270-4eb1-4b2b-bd58-f77200ceda0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:36.860268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-2735a270-4eb1-4b2b-bd58-f77200ceda0f None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:36.861818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-2735a270-4eb1-4b2b-bd58-f77200ceda0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:36.863649 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 15/29] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Jun 6 10:23:36 2022] POST /identity/v3/roles => generated 219 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:23:36.867334 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16333c0e-d39d-40c2-82d8-9ec048a9b785 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:36.867723 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16333c0e-d39d-40c2-82d8-9ec048a9b785 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:36.868443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16333c0e-d39d-40c2-82d8-9ec048a9b785 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:36.869875 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 16/30] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:23:36.870644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16333c0e-d39d-40c2-82d8-9ec048a9b785 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:36.899631 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:36.984219 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 15/31] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:23:36.986096 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882002dc-d6c5-4731-8a3e-d64e98c47470 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:36.986584 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882002dc-d6c5-4731-8a3e-d64e98c47470 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:36.987073 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882002dc-d6c5-4731-8a3e-d64e98c47470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:37.007123 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:37.037596 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-882002dc-d6c5-4731-8a3e-d64e98c47470 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:37.039458 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-882002dc-d6c5-4731-8a3e-d64e98c47470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:37.042236 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16333c0e-d39d-40c2-82d8-9ec048a9b785 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:37.103021 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c992a-9f43-4bcd-b199-5ed1f290b224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:37.103693 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c992a-9f43-4bcd-b199-5ed1f290b224 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:37.104002 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c992a-9f43-4bcd-b199-5ed1f290b224 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:37.104281 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c992a-9f43-4bcd-b199-5ed1f290b224 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:37.104995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c992a-9f43-4bcd-b199-5ed1f290b224 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:37.106583 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c992a-9f43-4bcd-b199-5ed1f290b224 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:37.141145 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 17/32] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jun 6 10:23:36 2022] POST /identity/v3/projects => generated 301 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:23:37.141944 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c992a-9f43-4bcd-b199-5ed1f290b224 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:37.153548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 16/33] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:23:37.198069 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882002dc-d6c5-4731-8a3e-d64e98c47470 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:37.267760 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:37.306843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c992a-9f43-4bcd-b199-5ed1f290b224 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:37.350175 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b5bc8-1685-4bbf-9dcd-c921524ae0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:37.351516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b5bc8-1685-4bbf-9dcd-c921524ae0f7 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:37.351887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b5bc8-1685-4bbf-9dcd-c921524ae0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:37.352172 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b5bc8-1685-4bbf-9dcd-c921524ae0f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:37.352952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b5bc8-1685-4bbf-9dcd-c921524ae0f7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:37.354420 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b5bc8-1685-4bbf-9dcd-c921524ae0f7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:37.377291 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 18/34] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Jun 6 10:23:37 2022] POST /identity/v3/roles => generated 225 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:23:37.399220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882002dc-d6c5-4731-8a3e-d64e98c47470 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:37.441327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924f9d1-ec76-4fb9-9d29-dbc52e470716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:37.442211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924f9d1-ec76-4fb9-9d29-dbc52e470716 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:37.442497 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924f9d1-ec76-4fb9-9d29-dbc52e470716 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:37.442775 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924f9d1-ec76-4fb9-9d29-dbc52e470716 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:37.445255 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:37.456769 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924f9d1-ec76-4fb9-9d29-dbc52e470716 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:23:37.465043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-1924f9d1-ec76-4fb9-9d29-dbc52e470716 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:23:37.465043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:23:37.465043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:23:37.465043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:23:37.465043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:23:37.465043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:23:37.465043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:23:37.465043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:23:37.465043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:23:37.465043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:23:37.465043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:23:37.465043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:23:37.465043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:23:37.465043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:23:37.465043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:23:37.465043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:23:37.465931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:23:37.465931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:23:37.465931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:23:37.465931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:23:37.465931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:23:37.465931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:23:37.465931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:23:37.465931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:23:37.465931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:23:37.465931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:23:37.465931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:23:37.465931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:23:37.465931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:23:37.465931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:23:37.465931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:23:37.465931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:23:37.466714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:23:37.466714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:23:37.466714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:23:37.466714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:23:37.466714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:23:37.466714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:23:37.466714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:23:37.466714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:23:37.466714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:23:37.466714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:23:37.466714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:23:37.466714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:23:37.466714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 17/35] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:23:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:23:37.474540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b5bc8-1685-4bbf-9dcd-c921524ae0f7 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:37.576680 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406be719-2f2e-4448-879c-e93b323a0e61 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:37.577006 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406be719-2f2e-4448-879c-e93b323a0e61 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:37.577639 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406be719-2f2e-4448-879c-e93b323a0e61 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:37.579090 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 18/36] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:23:37.592163 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995abc9e-f5fd-4654-a3a8-17fa12e51b98 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:37.592488 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995abc9e-f5fd-4654-a3a8-17fa12e51b98 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:37.593039 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995abc9e-f5fd-4654-a3a8-17fa12e51b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:37.600991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b5bc8-1685-4bbf-9dcd-c921524ae0f7 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:37.615865 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-995abc9e-f5fd-4654-a3a8-17fa12e51b98 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:37.618097 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-995abc9e-f5fd-4654-a3a8-17fa12e51b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:37.636935 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:37.643953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40aec8c-391a-4e79-88b3-0468b1560572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:37.644630 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40aec8c-391a-4e79-88b3-0468b1560572 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:37.644927 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40aec8c-391a-4e79-88b3-0468b1560572 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:37.645202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40aec8c-391a-4e79-88b3-0468b1560572 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:37.645974 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40aec8c-391a-4e79-88b3-0468b1560572 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:37.646459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40aec8c-391a-4e79-88b3-0468b1560572 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:37.652226 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40aec8c-391a-4e79-88b3-0468b1560572 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:37.668661 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 19/37] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:23:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:37.674338 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40aec8c-391a-4e79-88b3-0468b1560572 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:37.711000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 19/38] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:23:37.736610 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995abc9e-f5fd-4654-a3a8-17fa12e51b98 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:37.795048 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40aec8c-391a-4e79-88b3-0468b1560572 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:37.830243 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9c12c-980a-4180-b707-6f5730cbd445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:37.831127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9c12c-980a-4180-b707-6f5730cbd445 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:37.831417 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9c12c-980a-4180-b707-6f5730cbd445 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:37.831698 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9c12c-980a-4180-b707-6f5730cbd445 None admin] PATH_INFO: `/v3/users/dc953d765b8048419427028bb5821273` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:37.832950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a9c12c-980a-4180-b707-6f5730cbd445 None admin] RBAC: Authorizing `identity:get_user(user_id=dc953d765b8048419427028bb5821273)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:37.834406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a9c12c-980a-4180-b707-6f5730cbd445 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:37.837060 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 20/39] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:23:37 2022] GET /identity/v3/users/dc953d765b8048419427028bb5821273 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:23:37.842111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9c12c-980a-4180-b707-6f5730cbd445 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:37.869712 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995abc9e-f5fd-4654-a3a8-17fa12e51b98 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:37.895459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:37.905446 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:37.908340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b442a8e9-eb26-4929-8862-b58b591cc6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:37.908939 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442a8e9-eb26-4929-8862-b58b591cc6a5 None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:37.909226 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442a8e9-eb26-4929-8862-b58b591cc6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:37.909501 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442a8e9-eb26-4929-8862-b58b591cc6a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:37.910237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b442a8e9-eb26-4929-8862-b58b591cc6a5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:37.911630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b442a8e9-eb26-4929-8862-b58b591cc6a5 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:37.949159 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9c12c-980a-4180-b707-6f5730cbd445 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:37.985012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b7d6d-ca7d-4823-a244-206ab65c1ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:37.985802 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b7d6d-ca7d-4823-a244-206ab65c1ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:37.986086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b7d6d-ca7d-4823-a244-206ab65c1ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:37.986362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b7d6d-ca7d-4823-a244-206ab65c1ff8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:37.987524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b7d6d-ca7d-4823-a244-206ab65c1ff8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:37.989011 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b7d6d-ca7d-4823-a244-206ab65c1ff8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:37.990609 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 21/40] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:23:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:23:37.995618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5201a7c-7734-43a5-b7b4-791668ec31a1 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:37.995618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5201a7c-7734-43a5-b7b4-791668ec31a1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:37.995618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5201a7c-7734-43a5-b7b4-791668ec31a1 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:37.995618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 22/41] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:23:37.996300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5201a7c-7734-43a5-b7b4-791668ec31a1 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:38.037591 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 20/42] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Jun 6 10:23:37 2022] POST /identity/v3/roles => generated 223 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:23:38.039519 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf284b44-c236-478d-962c-c383e4d5c5eb None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:38.040147 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf284b44-c236-478d-962c-c383e4d5c5eb None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:38.040147 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf284b44-c236-478d-962c-c383e4d5c5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:38.058767 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-cf284b44-c236-478d-962c-c383e4d5c5eb None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:38.060426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-cf284b44-c236-478d-962c-c383e4d5c5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:38.124025 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5201a7c-7734-43a5-b7b4-791668ec31a1 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:38.140982 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 21/43] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:23:38.157485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:38.165742 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf284b44-c236-478d-962c-c383e4d5c5eb None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:38.166770 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5b3fc-9c83-4e4a-972b-565adf8ce4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:38.167667 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5b3fc-9c83-4e4a-972b-565adf8ce4bc None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:38.168042 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5b3fc-9c83-4e4a-972b-565adf8ce4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:38.168256 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5b3fc-9c83-4e4a-972b-565adf8ce4bc None admin] PATH_INFO: `/v3/domains/default/users/dc953d765b8048419427028bb5821273/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:38.171173 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c5b3fc-9c83-4e4a-972b-565adf8ce4bc None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=dc953d765b8048419427028bb5821273, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:38.172701 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c5b3fc-9c83-4e4a-972b-565adf8ce4bc None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:38.189366 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 23/44] 10.222.0.42 () {58 vars in 1361 bytes} [Mon Jun 6 10:23:37 2022] PUT /identity/v3/domains/default/users/dc953d765b8048419427028bb5821273/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:23:38.286367 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf284b44-c236-478d-962c-c383e4d5c5eb None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:38.302785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:38.326818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a935c5-538f-4163-ba97-1df15e8e397c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:38.327633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a935c5-538f-4163-ba97-1df15e8e397c None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:38.327949 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a935c5-538f-4163-ba97-1df15e8e397c None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:38.328227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a935c5-538f-4163-ba97-1df15e8e397c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:38.329454 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a935c5-538f-4163-ba97-1df15e8e397c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:38.331075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a935c5-538f-4163-ba97-1df15e8e397c None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:38.332798 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 22/45] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:23:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:23:38.338922 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5b3fc-9c83-4e4a-972b-565adf8ce4bc None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:38.443580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5b3fc-9c83-4e4a-972b-565adf8ce4bc None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:38.480143 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9896e92-8375-496e-8cfd-4fd04df1f6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:38.480804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9896e92-8375-496e-8cfd-4fd04df1f6f2 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:38.481094 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9896e92-8375-496e-8cfd-4fd04df1f6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:38.481371 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9896e92-8375-496e-8cfd-4fd04df1f6f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:38.482079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9896e92-8375-496e-8cfd-4fd04df1f6f2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:38.483529 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9896e92-8375-496e-8cfd-4fd04df1f6f2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:38.509044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 24/46] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jun 6 10:23:38 2022] POST /identity/v3/projects => generated 312 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:23:40.427547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f8b56-4839-4700-9196-e934d28466f6 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:40.428090 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f8b56-4839-4700-9196-e934d28466f6 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:40.428206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f8b56-4839-4700-9196-e934d28466f6 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:40.429526 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 23/47] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:23:40.435648 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5aaca-bd1e-45b3-b8e1-00c9e69a97a5 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:40.436219 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5aaca-bd1e-45b3-b8e1-00c9e69a97a5 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:40.436423 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5aaca-bd1e-45b3-b8e1-00c9e69a97a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:40.460827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-43a5aaca-bd1e-45b3-b8e1-00c9e69a97a5 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:40.463668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-43a5aaca-bd1e-45b3-b8e1-00c9e69a97a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:40.505090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:40.539955 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 25/48] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:23:40.562646 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f8b56-4839-4700-9196-e934d28466f6 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:40.661830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f8b56-4839-4700-9196-e934d28466f6 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:40.695485 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c96e8-ec23-462e-a303-6d53be8d6d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:40.696291 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c96e8-ec23-462e-a303-6d53be8d6d7c None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:40.696572 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c96e8-ec23-462e-a303-6d53be8d6d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:40.696843 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c96e8-ec23-462e-a303-6d53be8d6d7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:40.706683 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c96e8-ec23-462e-a303-6d53be8d6d7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:23:40.714957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-954c96e8-ec23-462e-a303-6d53be8d6d7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:23:40.714957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:23:40.714957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:23:40.714957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:23:40.714957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:23:40.714957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:23:40.714957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:23:40.714957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:23:40.714957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:23:40.714957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:23:40.714957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:23:40.714957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:23:40.714957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:23:40.714957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:23:40.714957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:23:40.714957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:23:40.715985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:23:40.715985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:23:40.715985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:23:40.715985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:23:40.715985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:23:40.715985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:23:40.715985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:23:40.715985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:23:40.715985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:23:40.715985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:23:40.715985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:23:40.715985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:23:40.715985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:23:40.715985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:23:40.715985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:23:40.715985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:23:40.716954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:23:40.716954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:23:40.716954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:23:40.716954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:23:40.716954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:23:40.716954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:23:40.716954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:23:40.716954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:23:40.716954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:23:40.716954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:23:40.716954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:23:40.716954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:23:40.716954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 24/49] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:23:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:23:40.721881 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5aaca-bd1e-45b3-b8e1-00c9e69a97a5 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:40.781084 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfbe36-9548-4a03-824b-7d46203bf835 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:40.781368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfbe36-9548-4a03-824b-7d46203bf835 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:40.781650 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfbe36-9548-4a03-824b-7d46203bf835 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:40.783006 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 25/50] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:23:40.788304 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a562b12-d133-4db7-80ec-89ace3b7b913 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:40.788304 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a562b12-d133-4db7-80ec-89ace3b7b913 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:40.788663 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a562b12-d133-4db7-80ec-89ace3b7b913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:40.813288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-0a562b12-d133-4db7-80ec-89ace3b7b913 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:40.814500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-0a562b12-d133-4db7-80ec-89ace3b7b913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:40.835920 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5aaca-bd1e-45b3-b8e1-00c9e69a97a5 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:40.865059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:40.875821 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892aae1a-3d29-4632-9bb8-e4f7226d2edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:40.876499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892aae1a-3d29-4632-9bb8-e4f7226d2edc None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:40.876788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892aae1a-3d29-4632-9bb8-e4f7226d2edc None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:40.877071 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892aae1a-3d29-4632-9bb8-e4f7226d2edc None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:40.877831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892aae1a-3d29-4632-9bb8-e4f7226d2edc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:40.878142 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892aae1a-3d29-4632-9bb8-e4f7226d2edc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:40.879551 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892aae1a-3d29-4632-9bb8-e4f7226d2edc None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:40.888366 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 26/51] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:23:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:40.893841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892aae1a-3d29-4632-9bb8-e4f7226d2edc None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:40.903423 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 26/52] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 10:23:40.924650 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:40.926032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a562b12-d133-4db7-80ec-89ace3b7b913 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:40.992491 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892aae1a-3d29-4632-9bb8-e4f7226d2edc None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:41.024483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a562b12-d133-4db7-80ec-89ace3b7b913 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:41.025898 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d78d1-7e4e-4b22-8a3d-d6edbd0f5fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:41.026775 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d78d1-7e4e-4b22-8a3d-d6edbd0f5fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:41.027057 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d78d1-7e4e-4b22-8a3d-d6edbd0f5fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:41.027331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d78d1-7e4e-4b22-8a3d-d6edbd0f5fe8 None admin] PATH_INFO: `/v3/users/dc953d765b8048419427028bb5821273` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:41.028565 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d78d1-7e4e-4b22-8a3d-d6edbd0f5fe8 None admin] RBAC: Authorizing `identity:get_user(user_id=dc953d765b8048419427028bb5821273)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:41.030022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d78d1-7e4e-4b22-8a3d-d6edbd0f5fe8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:41.031652 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 27/53] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:23:40 2022] GET /identity/v3/users/dc953d765b8048419427028bb5821273 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:23:41.036932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d78d1-7e4e-4b22-8a3d-d6edbd0f5fe8 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:41.060390 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096130ac-cfc8-41bd-a4bc-3959b6e910ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:41.061203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096130ac-cfc8-41bd-a4bc-3959b6e910ee None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:41.061492 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096130ac-cfc8-41bd-a4bc-3959b6e910ee None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:41.061762 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096130ac-cfc8-41bd-a4bc-3959b6e910ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:41.062923 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096130ac-cfc8-41bd-a4bc-3959b6e910ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:41.064410 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096130ac-cfc8-41bd-a4bc-3959b6e910ee None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:41.066174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 27/54] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:23:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:23:41.071961 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096130ac-cfc8-41bd-a4bc-3959b6e910ee None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:41.113668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:41.145167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:41.148787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d78d1-7e4e-4b22-8a3d-d6edbd0f5fe8 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:41.181482 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096130ac-cfc8-41bd-a4bc-3959b6e910ee None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:41.185711 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1b410-5ce5-4bdf-9c1a-c204768105a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:41.186542 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1b410-5ce5-4bdf-9c1a-c204768105a3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:41.186830 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1b410-5ce5-4bdf-9c1a-c204768105a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:41.187108 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1b410-5ce5-4bdf-9c1a-c204768105a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:41.188320 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd1b410-5ce5-4bdf-9c1a-c204768105a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:41.190195 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd1b410-5ce5-4bdf-9c1a-c204768105a3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:41.192036 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 28/55] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:23:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:23:41.198194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1b410-5ce5-4bdf-9c1a-c204768105a3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:41.215376 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd89cb9-5b5d-4748-832d-0add7ba91c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:41.216060 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd89cb9-5b5d-4748-832d-0add7ba91c61 None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:41.216346 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd89cb9-5b5d-4748-832d-0add7ba91c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:41.216623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd89cb9-5b5d-4748-832d-0add7ba91c61 None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:41.217316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd89cb9-5b5d-4748-832d-0add7ba91c61 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:41.218757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd89cb9-5b5d-4748-832d-0add7ba91c61 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:41.246432 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 28/56] 10.222.0.42 () {60 vars in 1140 bytes} [Mon Jun 6 10:23:41 2022] POST /identity/v3/projects => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:23:41.300080 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1b410-5ce5-4bdf-9c1a-c204768105a3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:41.336875 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b12871-cdc7-4bb0-9bd3-503ea8636d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:41.337554 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b12871-cdc7-4bb0-9bd3-503ea8636d85 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:41.337843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b12871-cdc7-4bb0-9bd3-503ea8636d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:41.338120 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b12871-cdc7-4bb0-9bd3-503ea8636d85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:41.338972 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b12871-cdc7-4bb0-9bd3-503ea8636d85 None admin] RBAC: Adding query filter params (role.id=0f929204eb7b4ea8859338ed2d3ff75b, scope.domain.id=default, user.id=dc953d765b8048419427028bb5821273) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:41.339289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b12871-cdc7-4bb0-9bd3-503ea8636d85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:41.340794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b12871-cdc7-4bb0-9bd3-503ea8636d85 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:41.347476 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 29/57] 10.222.0.42 () {56 vars in 1322 bytes} [Mon Jun 6 10:23:41 2022] GET /identity/v3/role_assignments?user.id=dc953d765b8048419427028bb5821273&scope.domain.id=default&role.id=0f929204eb7b4ea8859338ed2d3ff75b => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:23:43.489098 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685a613-f3d3-4c21-9650-19b7646d4b79 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:43.489611 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685a613-f3d3-4c21-9650-19b7646d4b79 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:43.489700 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685a613-f3d3-4c21-9650-19b7646d4b79 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:43.491098 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 29/58] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:23:43.496253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c302a-d309-4b8f-8d90-09ba4e5a1a1e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:43.496649 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c302a-d309-4b8f-8d90-09ba4e5a1a1e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:43.496824 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c302a-d309-4b8f-8d90-09ba4e5a1a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:43.513205 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-d03c302a-d309-4b8f-8d90-09ba4e5a1a1e None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:43.514383 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d03c302a-d309-4b8f-8d90-09ba4e5a1a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:43.540469 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:43.580055 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 30/59] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:23:43.605700 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685a613-f3d3-4c21-9650-19b7646d4b79 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:43.694890 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:43.705623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685a613-f3d3-4c21-9650-19b7646d4b79 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:43.737660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c037a217-e9d6-4986-877a-f81944720d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:43.738434 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037a217-e9d6-4986-877a-f81944720d0b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:43.738720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037a217-e9d6-4986-877a-f81944720d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:43.738996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037a217-e9d6-4986-877a-f81944720d0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:43.740189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c037a217-e9d6-4986-877a-f81944720d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:43.741655 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c037a217-e9d6-4986-877a-f81944720d0b None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:43.743240 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 30/60] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:23:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:43.748610 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c302a-d309-4b8f-8d90-09ba4e5a1a1e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:43.843394 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c302a-d309-4b8f-8d90-09ba4e5a1a1e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:43.876990 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de15c2a-0b07-4d8c-af5c-3f5931825980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:43.877649 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de15c2a-0b07-4d8c-af5c-3f5931825980 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:43.877940 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de15c2a-0b07-4d8c-af5c-3f5931825980 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:43.878224 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de15c2a-0b07-4d8c-af5c-3f5931825980 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:43.878924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de15c2a-0b07-4d8c-af5c-3f5931825980 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:43.880409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de15c2a-0b07-4d8c-af5c-3f5931825980 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:43.906505 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 31/61] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jun 6 10:23:43 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:23:47.508882 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c2c82-7e1f-4669-989f-77e9f0cc75f6 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:47.509380 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c2c82-7e1f-4669-989f-77e9f0cc75f6 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:47.509455 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c2c82-7e1f-4669-989f-77e9f0cc75f6 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:47.510858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 31/62] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:23:47.528027 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8bd62-2e6c-4e82-af11-7b94968cb3f3 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:47.528504 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8bd62-2e6c-4e82-af11-7b94968cb3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:47.528577 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8bd62-2e6c-4e82-af11-7b94968cb3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:47.552082 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-71f8bd62-2e6c-4e82-af11-7b94968cb3f3 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:47.553319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-71f8bd62-2e6c-4e82-af11-7b94968cb3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:47.578713 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:47.592042 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cd739-d007-446d-8418-ea499ae01454 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:47.592326 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cd739-d007-446d-8418-ea499ae01454 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:47.592830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cd739-d007-446d-8418-ea499ae01454 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:47.594667 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 32/63] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:23:47.604941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d9a61-b137-4a81-ba90-53dc4554b5ea None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:47.605499 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d9a61-b137-4a81-ba90-53dc4554b5ea None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:47.605983 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d9a61-b137-4a81-ba90-53dc4554b5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:47.640207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-346d9a61-b137-4a81-ba90-53dc4554b5ea None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:47.641329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-346d9a61-b137-4a81-ba90-53dc4554b5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:47.645495 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 32/64] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:23:47.702171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8bd62-2e6c-4e82-af11-7b94968cb3f3 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:47.753349 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 33/65] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:23:47.755869 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f32c71-6bb6-499b-a741-a98613315292 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:47.756372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f32c71-6bb6-499b-a741-a98613315292 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:47.756907 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f32c71-6bb6-499b-a741-a98613315292 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:47.758588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 34/66] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:23:47.764319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11221e59-f295-4d61-b6a8-480e4714e0f9 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:47.764779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11221e59-f295-4d61-b6a8-480e4714e0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:47.765208 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11221e59-f295-4d61-b6a8-480e4714e0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:47.790756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-11221e59-f295-4d61-b6a8-480e4714e0f9 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:47.792438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-11221e59-f295-4d61-b6a8-480e4714e0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:47.833201 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8bd62-2e6c-4e82-af11-7b94968cb3f3 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:47.869397 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 35/67] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 10:23:47.870634 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11221e59-f295-4d61-b6a8-480e4714e0f9 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:47.875093 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0236f0-f9de-4908-abec-46d645af1eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:47.876269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0236f0-f9de-4908-abec-46d645af1eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:47.876760 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0236f0-f9de-4908-abec-46d645af1eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:47.877282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0236f0-f9de-4908-abec-46d645af1eb4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:47.890567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0236f0-f9de-4908-abec-46d645af1eb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:23:47.896110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:47.902085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-fa0236f0-f9de-4908-abec-46d645af1eb4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:23:47.902085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:23:47.902085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:23:47.902085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:23:47.902085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:23:47.902085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:23:47.902085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:23:47.902085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:23:47.902085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:23:47.902085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:23:47.902085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:23:47.902085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:23:47.902085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:23:47.902085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:23:47.902085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:23:47.902085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:23:47.903167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:23:47.903167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:23:47.903167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:23:47.903167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:23:47.903167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:23:47.903167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:23:47.903167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:23:47.903167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:23:47.903167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:23:47.903167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:23:47.903167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:23:47.903167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:23:47.903167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:23:47.903167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:23:47.903167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:23:47.903167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:23:47.903977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:23:47.903977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:23:47.903977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:23:47.903977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:23:47.903977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:23:47.903977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:23:47.903977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:23:47.903977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:23:47.903977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:23:47.903977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:23:47.903977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:23:47.903977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:23:47.903977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 33/68] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:23:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:23:47.904621 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f4700-da73-4a09-839f-5d0608ec586f None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:47.905070 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f4700-da73-4a09-839f-5d0608ec586f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:47.905162 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f4700-da73-4a09-839f-5d0608ec586f None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:47.906551 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 34/69] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:23:47.910793 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f4700-da73-4a09-839f-5d0608ec586f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:47.920001 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:48.000228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11221e59-f295-4d61-b6a8-480e4714e0f9 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:48.017785 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f4700-da73-4a09-839f-5d0608ec586f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:48.036408 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928376a3-4014-4ac9-bfb3-1f9cd87910e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:48.037214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928376a3-4014-4ac9-bfb3-1f9cd87910e2 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:48.038184 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928376a3-4014-4ac9-bfb3-1f9cd87910e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:48.038878 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928376a3-4014-4ac9-bfb3-1f9cd87910e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:48.048893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928376a3-4014-4ac9-bfb3-1f9cd87910e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:23:48.053779 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ec3dd-e442-4032-a216-0a0e8f5d8ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:48.054597 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ec3dd-e442-4032-a216-0a0e8f5d8ced None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:48.054859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ec3dd-e442-4032-a216-0a0e8f5d8ced None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:48.055131 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ec3dd-e442-4032-a216-0a0e8f5d8ced None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:48.057926 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-928376a3-4014-4ac9-bfb3-1f9cd87910e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:23:48.057926 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:23:48.057926 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:23:48.057926 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:23:48.057926 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:23:48.057926 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:23:48.057926 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:23:48.057926 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:23:48.057926 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:23:48.057926 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:23:48.057926 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:23:48.057926 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:23:48.057926 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:23:48.057926 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:23:48.057926 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:23:48.057926 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:23:48.058765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:23:48.058765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:23:48.058765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:23:48.058765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:23:48.058765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:23:48.058765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:23:48.058765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:23:48.058765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:23:48.058765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:23:48.058765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:23:48.058765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:23:48.058765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:23:48.058765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:23:48.058765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:23:48.058765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:23:48.058765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:23:48.059500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:23:48.059500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:23:48.059500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:23:48.059500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:23:48.059500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:23:48.059500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:23:48.059500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:23:48.059500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:23:48.059500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:23:48.059500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:23:48.059500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:23:48.059500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:23:48.059500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 36/70] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:23:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:23:48.060661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928376a3-4014-4ac9-bfb3-1f9cd87910e2 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:48.073099 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ec3dd-e442-4032-a216-0a0e8f5d8ced None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:23:48.080853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-e15ec3dd-e442-4032-a216-0a0e8f5d8ced None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:23:48.080853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:23:48.080853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:23:48.080853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:23:48.080853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:23:48.080853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:23:48.080853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:23:48.080853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:23:48.080853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:23:48.080853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:23:48.080853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:23:48.080853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:23:48.080853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:23:48.080853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:23:48.080853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:23:48.080853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:23:48.081680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:23:48.081680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:23:48.081680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:23:48.081680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:23:48.081680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:23:48.081680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:23:48.081680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:23:48.081680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:23:48.081680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:23:48.081680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:23:48.081680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:23:48.081680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:23:48.081680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:23:48.081680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:23:48.081680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:23:48.081680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:23:48.082445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:23:48.082445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:23:48.082445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:23:48.082445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:23:48.082445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:23:48.082445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:23:48.082445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:23:48.082445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:23:48.082445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:23:48.082445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:23:48.082445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:23:48.082445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:23:48.082445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 35/71] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:23:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:23:48.083514 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98774e60-6be0-4665-9426-0564fc1db10c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:48.083514 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98774e60-6be0-4665-9426-0564fc1db10c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:48.083815 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98774e60-6be0-4665-9426-0564fc1db10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:48.100981 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-98774e60-6be0-4665-9426-0564fc1db10c None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:48.102169 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-98774e60-6be0-4665-9426-0564fc1db10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:48.155318 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:48.168705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:48.175711 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 36/72] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:23:48.176379 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98774e60-6be0-4665-9426-0564fc1db10c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:48.184645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928376a3-4014-4ac9-bfb3-1f9cd87910e2 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:48.223517 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46524b5f-b731-47b3-86ca-7046fc52e5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:48.224215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46524b5f-b731-47b3-86ca-7046fc52e5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:48.224503 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46524b5f-b731-47b3-86ca-7046fc52e5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:48.224783 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46524b5f-b731-47b3-86ca-7046fc52e5c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:48.226251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46524b5f-b731-47b3-86ca-7046fc52e5c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:48.226559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46524b5f-b731-47b3-86ca-7046fc52e5c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:48.227974 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46524b5f-b731-47b3-86ca-7046fc52e5c2 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:48.236255 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 37/73] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:23:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:48.241248 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46524b5f-b731-47b3-86ca-7046fc52e5c2 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:48.287202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98774e60-6be0-4665-9426-0564fc1db10c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:48.320900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa130d-3720-4f8e-8042-193c65fd1bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:48.321563 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa130d-3720-4f8e-8042-193c65fd1bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:48.321848 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa130d-3720-4f8e-8042-193c65fd1bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:48.322131 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa130d-3720-4f8e-8042-193c65fd1bd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:48.322876 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa130d-3720-4f8e-8042-193c65fd1bd6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:48.323186 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa130d-3720-4f8e-8042-193c65fd1bd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:48.324607 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa130d-3720-4f8e-8042-193c65fd1bd6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:48.332265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 37/74] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:23:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:48.333448 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7453e-fd04-4a03-837e-cb0ba60752fe None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:48.333736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7453e-fd04-4a03-837e-cb0ba60752fe None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:48.334013 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7453e-fd04-4a03-837e-cb0ba60752fe None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:48.335820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 38/75] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:23:48.336520 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7453e-fd04-4a03-837e-cb0ba60752fe None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:48.345079 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46524b5f-b731-47b3-86ca-7046fc52e5c2 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:48.378949 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3413d-44c8-4cbe-8f10-eb08d17fec28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:48.379618 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3413d-44c8-4cbe-8f10-eb08d17fec28 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:48.379940 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3413d-44c8-4cbe-8f10-eb08d17fec28 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:48.380219 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3413d-44c8-4cbe-8f10-eb08d17fec28 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:48.380968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3413d-44c8-4cbe-8f10-eb08d17fec28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:48.381276 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3413d-44c8-4cbe-8f10-eb08d17fec28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:48.382676 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3413d-44c8-4cbe-8f10-eb08d17fec28 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:48.390979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 38/76] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:23:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:48.391891 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3413d-44c8-4cbe-8f10-eb08d17fec28 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:48.397126 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:48.400465 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:48.439376 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7453e-fd04-4a03-837e-cb0ba60752fe None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:48.471900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11765b52-341f-4132-976f-e0e5d9576f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:48.472696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11765b52-341f-4132-976f-e0e5d9576f23 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:48.472981 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11765b52-341f-4132-976f-e0e5d9576f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:48.473263 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11765b52-341f-4132-976f-e0e5d9576f23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:48.484691 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11765b52-341f-4132-976f-e0e5d9576f23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:23:48.493177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-11765b52-341f-4132-976f-e0e5d9576f23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:23:48.493177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:23:48.493177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:23:48.493177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:23:48.493177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:23:48.493177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:23:48.493177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:23:48.493177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:23:48.493177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:23:48.493177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:23:48.493177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:23:48.493177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:23:48.493177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:23:48.493177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:23:48.493177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:23:48.493177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:23:48.494116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:23:48.494116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:23:48.494116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:23:48.494116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:23:48.494116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:23:48.494116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:23:48.494116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:23:48.494116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:23:48.494116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:23:48.494116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:23:48.494116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:23:48.494116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:23:48.494116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:23:48.494116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:23:48.494116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:23:48.494116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:23:48.494889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:23:48.494889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:23:48.494889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:23:48.494889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:23:48.494889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:23:48.494889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:23:48.494889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:23:48.494889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:23:48.494889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:23:48.494889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:23:48.494889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:23:48.494889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:23:48.494889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 39/77] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:23:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:23:48.495521 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11765b52-341f-4132-976f-e0e5d9576f23 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:48.504501 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3413d-44c8-4cbe-8f10-eb08d17fec28 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:48.539695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b93fc0-5c78-4c08-91ec-ae40354ec01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:48.540595 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b93fc0-5c78-4c08-91ec-ae40354ec01e None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:48.540880 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b93fc0-5c78-4c08-91ec-ae40354ec01e None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:48.541164 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b93fc0-5c78-4c08-91ec-ae40354ec01e None admin] PATH_INFO: `/v3/users/709a0882f46a4bcaa3f47007f20cfaa1` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:48.556352 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b93fc0-5c78-4c08-91ec-ae40354ec01e None admin] RBAC: Authorizing `identity:get_user(user_id=709a0882f46a4bcaa3f47007f20cfaa1)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:48.557780 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b93fc0-5c78-4c08-91ec-ae40354ec01e None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:48.559428 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 39/78] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:23:48 2022] GET /identity/v3/users/709a0882f46a4bcaa3f47007f20cfaa1 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:48.560617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7d74c-7b0c-41b2-92c6-da794acf33fa None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:48.560900 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7d74c-7b0c-41b2-92c6-da794acf33fa None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:48.561185 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7d74c-7b0c-41b2-92c6-da794acf33fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:48.577912 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-61d7d74c-7b0c-41b2-92c6-da794acf33fa None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:48.579104 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-61d7d74c-7b0c-41b2-92c6-da794acf33fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:48.594045 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11765b52-341f-4132-976f-e0e5d9576f23 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:48.613437 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:48.628806 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c016fed-3715-4b08-9d13-842dc534b4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:48.629688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c016fed-3715-4b08-9d13-842dc534b4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:48.629972 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c016fed-3715-4b08-9d13-842dc534b4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:48.630247 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c016fed-3715-4b08-9d13-842dc534b4f8 None admin] PATH_INFO: `/v3/users/709a0882f46a4bcaa3f47007f20cfaa1` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:48.631481 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c016fed-3715-4b08-9d13-842dc534b4f8 None admin] RBAC: Authorizing `identity:get_user(user_id=709a0882f46a4bcaa3f47007f20cfaa1)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:48.632982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c016fed-3715-4b08-9d13-842dc534b4f8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:48.634606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 40/79] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:23:48 2022] GET /identity/v3/users/709a0882f46a4bcaa3f47007f20cfaa1 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:48.635282 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:48.635389 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c016fed-3715-4b08-9d13-842dc534b4f8 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:48.650408 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 40/80] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:23:48.656645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7d74c-7b0c-41b2-92c6-da794acf33fa None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:48.737306 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c016fed-3715-4b08-9d13-842dc534b4f8 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:48.754978 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7d74c-7b0c-41b2-92c6-da794acf33fa None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:48.769925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531f2a1-f073-4e34-a66b-e39180136047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:48.770796 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531f2a1-f073-4e34-a66b-e39180136047 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:48.771080 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531f2a1-f073-4e34-a66b-e39180136047 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:48.771355 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531f2a1-f073-4e34-a66b-e39180136047 None admin] PATH_INFO: `/v3/users/dc953d765b8048419427028bb5821273` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:48.773167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0531f2a1-f073-4e34-a66b-e39180136047 None admin] RBAC: Authorizing `identity:get_user(user_id=dc953d765b8048419427028bb5821273)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:48.774882 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0531f2a1-f073-4e34-a66b-e39180136047 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:48.776531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 41/81] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:23:48 2022] GET /identity/v3/users/dc953d765b8048419427028bb5821273 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:48.777285 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531f2a1-f073-4e34-a66b-e39180136047 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:48.789377 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463fd92-7429-4300-bdc5-8d8a99812c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:48.790048 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463fd92-7429-4300-bdc5-8d8a99812c3a None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:48.790333 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463fd92-7429-4300-bdc5-8d8a99812c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:48.790611 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463fd92-7429-4300-bdc5-8d8a99812c3a None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:48.791350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463fd92-7429-4300-bdc5-8d8a99812c3a None admin] RBAC: Adding query filter params (name=member) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:48.791658 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463fd92-7429-4300-bdc5-8d8a99812c3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:48.793064 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463fd92-7429-4300-bdc5-8d8a99812c3a None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:48.801396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 41/82] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:23:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:48.801396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463fd92-7429-4300-bdc5-8d8a99812c3a None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:48.835414 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:48.861352 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:48.880288 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531f2a1-f073-4e34-a66b-e39180136047 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:48.906210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463fd92-7429-4300-bdc5-8d8a99812c3a None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:48.914957 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929c32a-e18c-4d90-9192-b77a934bc35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:48.915922 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929c32a-e18c-4d90-9192-b77a934bc35a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:48.916215 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929c32a-e18c-4d90-9192-b77a934bc35a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:48.916496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929c32a-e18c-4d90-9192-b77a934bc35a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:48.939733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377b16c-8b17-44ab-9033-e28b3c0a6644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:48.940397 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377b16c-8b17-44ab-9033-e28b3c0a6644 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:48.940685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377b16c-8b17-44ab-9033-e28b3c0a6644 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:48.940962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377b16c-8b17-44ab-9033-e28b3c0a6644 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:48.982706 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929c32a-e18c-4d90-9192-b77a934bc35a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:48.984222 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929c32a-e18c-4d90-9192-b77a934bc35a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:49.007125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377b16c-8b17-44ab-9033-e28b3c0a6644 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:49.008546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377b16c-8b17-44ab-9033-e28b3c0a6644 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:49.039612 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:49.063255 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:49.083837 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 42/83] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:23:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:23:49.084711 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929c32a-e18c-4d90-9192-b77a934bc35a None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:49.111942 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 42/84] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:23:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:23:49.112869 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377b16c-8b17-44ab-9033-e28b3c0a6644 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:49.187267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929c32a-e18c-4d90-9192-b77a934bc35a None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:49.211028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377b16c-8b17-44ab-9033-e28b3c0a6644 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:49.224552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377e546-0875-41ab-a177-33ff364d0ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:49.226012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377e546-0875-41ab-a177-33ff364d0ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:49.226323 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377e546-0875-41ab-a177-33ff364d0ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:49.226606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377e546-0875-41ab-a177-33ff364d0ce7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:49.227825 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377e546-0875-41ab-a177-33ff364d0ce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:49.229307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377e546-0875-41ab-a177-33ff364d0ce7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:49.230941 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 43/85] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:23:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:49.232706 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377e546-0875-41ab-a177-33ff364d0ce7 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:49.246227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f07f03-0a5a-4b73-9067-62e939075bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:49.246879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f07f03-0a5a-4b73-9067-62e939075bbc None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:49.247164 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f07f03-0a5a-4b73-9067-62e939075bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:49.247439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f07f03-0a5a-4b73-9067-62e939075bbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:49.257429 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:49.272520 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:49.317641 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f07f03-0a5a-4b73-9067-62e939075bbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:49.319711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f07f03-0a5a-4b73-9067-62e939075bbc None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:49.331458 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377e546-0875-41ab-a177-33ff364d0ce7 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:49.364659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b1aa5-2b63-4507-a09c-f7990fc8139f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:49.366044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b1aa5-2b63-4507-a09c-f7990fc8139f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:49.366510 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b1aa5-2b63-4507-a09c-f7990fc8139f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:49.366884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b1aa5-2b63-4507-a09c-f7990fc8139f None admin] PATH_INFO: `/v3/users/709a0882f46a4bcaa3f47007f20cfaa1` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:49.368138 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b1aa5-2b63-4507-a09c-f7990fc8139f None admin] RBAC: Authorizing `identity:get_user(user_id=709a0882f46a4bcaa3f47007f20cfaa1)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:49.369580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b1aa5-2b63-4507-a09c-f7990fc8139f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:49.371206 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 44/86] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:23:49 2022] GET /identity/v3/users/709a0882f46a4bcaa3f47007f20cfaa1 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:49.371985 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b1aa5-2b63-4507-a09c-f7990fc8139f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:49.423242 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 43/87] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:23:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:23:49.424355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f07f03-0a5a-4b73-9067-62e939075bbc None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:49.466332 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:49.476238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b1aa5-2b63-4507-a09c-f7990fc8139f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:49.481761 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:49.517295 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6badb-dfcc-488f-8a2c-2938ec0dbf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:49.518190 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6badb-dfcc-488f-8a2c-2938ec0dbf6d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:49.518472 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6badb-dfcc-488f-8a2c-2938ec0dbf6d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:49.518745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6badb-dfcc-488f-8a2c-2938ec0dbf6d None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:49.519952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6badb-dfcc-488f-8a2c-2938ec0dbf6d None admin] RBAC: Authorizing `identity:get_project(project_id=0efa83f60031447ab87be4f6f1af6165)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:49.521390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6badb-dfcc-488f-8a2c-2938ec0dbf6d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:49.523092 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 45/88] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:23:49 2022] GET /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:49.525107 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6badb-dfcc-488f-8a2c-2938ec0dbf6d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:49.531671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f07f03-0a5a-4b73-9067-62e939075bbc None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:49.565467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10eb204-591c-416c-809f-59623386c70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:49.566329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10eb204-591c-416c-809f-59623386c70f None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:49.566614 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10eb204-591c-416c-809f-59623386c70f None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:49.566890 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10eb204-591c-416c-809f-59623386c70f None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:49.568097 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10eb204-591c-416c-809f-59623386c70f None admin] RBAC: Authorizing `identity:get_project(project_id=0efa83f60031447ab87be4f6f1af6165)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:49.570136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10eb204-591c-416c-809f-59623386c70f None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:49.571860 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 44/89] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:23:49 2022] GET /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:49.572625 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10eb204-591c-416c-809f-59623386c70f None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:49.628516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6badb-dfcc-488f-8a2c-2938ec0dbf6d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:49.662256 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108f085-1580-430e-9ad5-d241507c5265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:49.663485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108f085-1580-430e-9ad5-d241507c5265 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:49.663936 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108f085-1580-430e-9ad5-d241507c5265 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:49.664213 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108f085-1580-430e-9ad5-d241507c5265 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:49.664920 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108f085-1580-430e-9ad5-d241507c5265 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:49.666389 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108f085-1580-430e-9ad5-d241507c5265 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:49.670215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10eb204-591c-416c-809f-59623386c70f None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:49.685945 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 46/90] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jun 6 10:23:49 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:23:49.686750 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108f085-1580-430e-9ad5-d241507c5265 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:49.706645 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:49.707574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:49.727206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5ff2e-ea64-4c3d-8e70-e32711f158c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:49.728022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5ff2e-ea64-4c3d-8e70-e32711f158c8 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:49.728315 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5ff2e-ea64-4c3d-8e70-e32711f158c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:49.728711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5ff2e-ea64-4c3d-8e70-e32711f158c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:49.803439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108f085-1580-430e-9ad5-d241507c5265 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:49.807746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5ff2e-ea64-4c3d-8e70-e32711f158c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:49.809293 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5ff2e-ea64-4c3d-8e70-e32711f158c8 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:49.842709 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45f5f0-291a-4686-a9dc-313b27d43adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:49.843617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45f5f0-291a-4686-a9dc-313b27d43adc None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:49.843951 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45f5f0-291a-4686-a9dc-313b27d43adc None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:49.844233 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45f5f0-291a-4686-a9dc-313b27d43adc None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:49.845594 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc45f5f0-291a-4686-a9dc-313b27d43adc None admin] RBAC: Authorizing `identity:get_project(project_id=0efa83f60031447ab87be4f6f1af6165)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:49.847143 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc45f5f0-291a-4686-a9dc-313b27d43adc None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:49.854457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 47/91] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:23:49 2022] GET /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:49.854457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45f5f0-291a-4686-a9dc-313b27d43adc None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:49.934176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 45/92] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:23:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:23:49.936096 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5ff2e-ea64-4c3d-8e70-e32711f158c8 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:49.955383 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:49.964424 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:49.979071 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45f5f0-291a-4686-a9dc-313b27d43adc None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:50.026590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ab70f-b0af-4b17-b64e-dae7d1927674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:50.027246 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ab70f-b0af-4b17-b64e-dae7d1927674 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:50.027532 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ab70f-b0af-4b17-b64e-dae7d1927674 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:50.027833 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ab70f-b0af-4b17-b64e-dae7d1927674 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:50.029179 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ab70f-b0af-4b17-b64e-dae7d1927674 None admin] RBAC: Adding query filter params (role.id=6bcb06bf4aa4442ebcfebeadf1af41de, scope.project.id=0efa83f60031447ab87be4f6f1af6165, user.id=709a0882f46a4bcaa3f47007f20cfaa1) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:50.029179 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ab70f-b0af-4b17-b64e-dae7d1927674 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:50.030424 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ab70f-b0af-4b17-b64e-dae7d1927674 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:50.036796 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 48/93] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:23:49 2022] GET /identity/v3/role_assignments?user.id=709a0882f46a4bcaa3f47007f20cfaa1&scope.project.id=0efa83f60031447ab87be4f6f1af6165&role.id=6bcb06bf4aa4442ebcfebeadf1af41de => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:50.038654 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ab70f-b0af-4b17-b64e-dae7d1927674 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:50.069292 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5ff2e-ea64-4c3d-8e70-e32711f158c8 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:50.120896 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30245fb2-08c5-417b-8bdb-68102e9012e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:50.121584 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30245fb2-08c5-417b-8bdb-68102e9012e4 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:50.121875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30245fb2-08c5-417b-8bdb-68102e9012e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:50.122466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30245fb2-08c5-417b-8bdb-68102e9012e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:50.124036 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30245fb2-08c5-417b-8bdb-68102e9012e4 None admin] RBAC: Adding query filter params (role.id=f214f8dbc3354cfc81fa2c2f116980e0, scope.project.id=0efa83f60031447ab87be4f6f1af6165, user.id=709a0882f46a4bcaa3f47007f20cfaa1) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:50.124375 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30245fb2-08c5-417b-8bdb-68102e9012e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:50.125855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30245fb2-08c5-417b-8bdb-68102e9012e4 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:50.141061 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 46/94] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:23:49 2022] GET /identity/v3/role_assignments?user.id=709a0882f46a4bcaa3f47007f20cfaa1&scope.project.id=0efa83f60031447ab87be4f6f1af6165&role.id=f214f8dbc3354cfc81fa2c2f116980e0 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:50.142760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30245fb2-08c5-417b-8bdb-68102e9012e4 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:50.179567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ab70f-b0af-4b17-b64e-dae7d1927674 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:50.226223 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d9678-6683-41da-9848-8a9421c595b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:50.227246 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d9678-6683-41da-9848-8a9421c595b4 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:50.227715 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d9678-6683-41da-9848-8a9421c595b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:50.228206 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d9678-6683-41da-9848-8a9421c595b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:50.229257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d9678-6683-41da-9848-8a9421c595b4 None admin] RBAC: Adding query filter params (role.id=0f929204eb7b4ea8859338ed2d3ff75b, scope.project.id=0efa83f60031447ab87be4f6f1af6165, user.id=dc953d765b8048419427028bb5821273) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:50.229790 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d9678-6683-41da-9848-8a9421c595b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:50.231515 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d9678-6683-41da-9848-8a9421c595b4 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:50.238661 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 49/95] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:23:50 2022] GET /identity/v3/role_assignments?user.id=dc953d765b8048419427028bb5821273&scope.project.id=0efa83f60031447ab87be4f6f1af6165&role.id=0f929204eb7b4ea8859338ed2d3ff75b => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:50.251719 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:50.274843 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30245fb2-08c5-417b-8bdb-68102e9012e4 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:50.323747 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a9fee-66f8-4662-b546-562f96057856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:50.324584 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a9fee-66f8-4662-b546-562f96057856 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:50.324869 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a9fee-66f8-4662-b546-562f96057856 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:50.325142 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a9fee-66f8-4662-b546-562f96057856 None admin] PATH_INFO: `/v3/projects/09ed03f605ac4be88166564d72a5e9e3` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:50.326473 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a9fee-66f8-4662-b546-562f96057856 None admin] RBAC: Authorizing `identity:get_project(project_id=09ed03f605ac4be88166564d72a5e9e3)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:50.328023 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a9fee-66f8-4662-b546-562f96057856 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:50.330169 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 47/96] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:23:50 2022] GET /identity/v3/projects/09ed03f605ac4be88166564d72a5e9e3 => generated 312 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:50.336558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d9678-6683-41da-9848-8a9421c595b4 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:50.364906 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:50.489165 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d9678-6683-41da-9848-8a9421c595b4 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:50.538480 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81021f-c49d-4569-a8bf-a366dc405bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:50.539155 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81021f-c49d-4569-a8bf-a366dc405bda None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:50.539447 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81021f-c49d-4569-a8bf-a366dc405bda None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:50.539725 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81021f-c49d-4569-a8bf-a366dc405bda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:50.540601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81021f-c49d-4569-a8bf-a366dc405bda None admin] RBAC: Adding query filter params (role.id=6bcb06bf4aa4442ebcfebeadf1af41de, scope.project.id=09ed03f605ac4be88166564d72a5e9e3, user.id=709a0882f46a4bcaa3f47007f20cfaa1) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:50.540920 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81021f-c49d-4569-a8bf-a366dc405bda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:50.542396 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81021f-c49d-4569-a8bf-a366dc405bda None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:50.548629 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 50/97] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:23:50 2022] GET /identity/v3/role_assignments?user.id=709a0882f46a4bcaa3f47007f20cfaa1&scope.project.id=09ed03f605ac4be88166564d72a5e9e3&role.id=6bcb06bf4aa4442ebcfebeadf1af41de => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:52.907603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec5108-a71d-4f38-8532-9afa4254ec90 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:52.908690 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec5108-a71d-4f38-8532-9afa4254ec90 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:52.909175 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec5108-a71d-4f38-8532-9afa4254ec90 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:52.910736 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 48/98] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:23:52.929279 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7534d-070e-4b14-87f1-9b880375b13e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:52.930302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7534d-070e-4b14-87f1-9b880375b13e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:52.930774 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7534d-070e-4b14-87f1-9b880375b13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:52.957050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-a9d7534d-070e-4b14-87f1-9b880375b13e None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:52.958606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a9d7534d-070e-4b14-87f1-9b880375b13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:53.050114 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 51/99] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:23:53.093211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec5108-a71d-4f38-8532-9afa4254ec90 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:53.245582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec5108-a71d-4f38-8532-9afa4254ec90 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:53.250512 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cea61-5e7f-440b-90cd-860fc797f3dd None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:53.251066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cea61-5e7f-440b-90cd-860fc797f3dd None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:53.251532 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cea61-5e7f-440b-90cd-860fc797f3dd None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:53.253281 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 52/100] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:23:53.260439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb6301-84a0-4b44-b7d7-199b997e5524 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:53.261006 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb6301-84a0-4b44-b7d7-199b997e5524 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:53.261478 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb6301-84a0-4b44-b7d7-199b997e5524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:53.288024 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:53.295959 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-80eb6301-84a0-4b44-b7d7-199b997e5524 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:53.297186 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-80eb6301-84a0-4b44-b7d7-199b997e5524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:53.301113 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782518e-df7c-4d94-adb4-6bc753d5fa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:53.301934 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782518e-df7c-4d94-adb4-6bc753d5fa69 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:53.302224 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782518e-df7c-4d94-adb4-6bc753d5fa69 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:53.302508 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782518e-df7c-4d94-adb4-6bc753d5fa69 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:53.325247 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782518e-df7c-4d94-adb4-6bc753d5fa69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:23:53.338342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-d782518e-df7c-4d94-adb4-6bc753d5fa69 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 10:23:53.338342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:23:53.338342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:23:53.338342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:23:53.338342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:23:53.338342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:23:53.338342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:23:53.338342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:23:53.338342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:23:53.338342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:23:53.338342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:23:53.338342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:23:53.338342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:23:53.338342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:23:53.338342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:23:53.338342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:23:53.339210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:23:53.339210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:23:53.339210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:23:53.339210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:23:53.339210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:23:53.339210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:23:53.339210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:23:53.339210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:23:53.339210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:23:53.339210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:23:53.339210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:23:53.339210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:23:53.339210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:23:53.339210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:23:53.339210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:23:53.339210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:23:53.340143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:23:53.340143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:23:53.340143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:23:53.340143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:23:53.340143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:23:53.340143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:23:53.340143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:23:53.340143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:23:53.340143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:23:53.340143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:23:53.340143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 10:23:53.340143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:23:53.340143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 49/101] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:23:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:23:53.346840 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782518e-df7c-4d94-adb4-6bc753d5fa69 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:53.372437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:53.497571 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 53/102] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 10:23:53.524791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad6d6b-01e2-497a-b3e2-839ff27a952d None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:53.525417 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad6d6b-01e2-497a-b3e2-839ff27a952d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:53.525952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad6d6b-01e2-497a-b3e2-839ff27a952d None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:53.527544 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 54/103] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:23:53.536957 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b882e0c-2afe-40fd-81ba-62237764034e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:53.537603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b882e0c-2afe-40fd-81ba-62237764034e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:53.538084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b882e0c-2afe-40fd-81ba-62237764034e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:53.546132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782518e-df7c-4d94-adb4-6bc753d5fa69 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:53.562260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-5b882e0c-2afe-40fd-81ba-62237764034e None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:53.563470 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5b882e0c-2afe-40fd-81ba-62237764034e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:53.598551 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60498cc0-74fb-4074-87a2-42edaa1ffdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:53.599535 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60498cc0-74fb-4074-87a2-42edaa1ffdec None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:53.600027 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60498cc0-74fb-4074-87a2-42edaa1ffdec None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:53.600445 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60498cc0-74fb-4074-87a2-42edaa1ffdec None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:53.601250 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60498cc0-74fb-4074-87a2-42edaa1ffdec None admin] RBAC: Adding query filter params (name=reader) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:53.601561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60498cc0-74fb-4074-87a2-42edaa1ffdec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:53.602948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60498cc0-74fb-4074-87a2-42edaa1ffdec None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:53.613980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 50/104] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:23:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:53.616733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60498cc0-74fb-4074-87a2-42edaa1ffdec None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:53.643200 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 55/105] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 10:23:53.644885 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3195a4-a5fc-417b-a481-3ceccc5a1d72 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:53.645324 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3195a4-a5fc-417b-a481-3ceccc5a1d72 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:53.645741 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3195a4-a5fc-417b-a481-3ceccc5a1d72 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:53.652359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 56/106] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:23:53.653412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3195a4-a5fc-417b-a481-3ceccc5a1d72 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:53.732683 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:53.758861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60498cc0-74fb-4074-87a2-42edaa1ffdec None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:53.766852 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:53.798932 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d72b2b-7e0c-4dcf-9281-758dcd325aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:53.800006 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d72b2b-7e0c-4dcf-9281-758dcd325aff None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:53.800458 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d72b2b-7e0c-4dcf-9281-758dcd325aff None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:53.800877 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d72b2b-7e0c-4dcf-9281-758dcd325aff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:53.802278 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d72b2b-7e0c-4dcf-9281-758dcd325aff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:53.803965 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d72b2b-7e0c-4dcf-9281-758dcd325aff None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:53.805947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 51/107] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:23:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:53.807326 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32de3d9-900b-4669-b7e7-4e3cd9efd8ec None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:53.807794 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32de3d9-900b-4669-b7e7-4e3cd9efd8ec None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:53.808215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32de3d9-900b-4669-b7e7-4e3cd9efd8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:53.818972 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3195a4-a5fc-417b-a481-3ceccc5a1d72 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:53.831446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-c32de3d9-900b-4669-b7e7-4e3cd9efd8ec None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:53.833608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-c32de3d9-900b-4669-b7e7-4e3cd9efd8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:53.861699 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc730a-3eb4-4760-82bd-72591762e7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:53.863121 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc730a-3eb4-4760-82bd-72591762e7e1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:53.863406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc730a-3eb4-4760-82bd-72591762e7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:53.863684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc730a-3eb4-4760-82bd-72591762e7e1 None admin] PATH_INFO: `/v3/users/d518e1cdf98a415ebb623b7286bc4086` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:53.882836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc730a-3eb4-4760-82bd-72591762e7e1 None admin] RBAC: Authorizing `identity:get_user(user_id=d518e1cdf98a415ebb623b7286bc4086)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:53.885139 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc730a-3eb4-4760-82bd-72591762e7e1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:53.889374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 57/108] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:23:53 2022] GET /identity/v3/users/d518e1cdf98a415ebb623b7286bc4086 => generated 289 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:53.890348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc730a-3eb4-4760-82bd-72591762e7e1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:53.926120 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 52/109] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:23:53.931857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32de3d9-900b-4669-b7e7-4e3cd9efd8ec None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:54.009906 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:54.029553 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc730a-3eb4-4760-82bd-72591762e7e1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:54.038148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32de3d9-900b-4669-b7e7-4e3cd9efd8ec None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:54.056663 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:54.069137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4067933-a341-4762-8195-c69c4ba2d146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:54.069968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4067933-a341-4762-8195-c69c4ba2d146 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:54.070263 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4067933-a341-4762-8195-c69c4ba2d146 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:54.070541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4067933-a341-4762-8195-c69c4ba2d146 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:54.077408 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c960c-1bca-4d75-a929-788fd5727240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:54.078089 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c960c-1bca-4d75-a929-788fd5727240 None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:54.078392 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c960c-1bca-4d75-a929-788fd5727240 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:54.078731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c960c-1bca-4d75-a929-788fd5727240 None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:54.079464 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c960c-1bca-4d75-a929-788fd5727240 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:54.081021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c960c-1bca-4d75-a929-788fd5727240 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:54.082897 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4067933-a341-4762-8195-c69c4ba2d146 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:23:54.091952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-b4067933-a341-4762-8195-c69c4ba2d146 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:23:54.091952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:23:54.091952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:23:54.091952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:23:54.091952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:23:54.091952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:23:54.091952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:23:54.091952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:23:54.091952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:23:54.091952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:23:54.091952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:23:54.091952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:23:54.091952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:23:54.091952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:23:54.091952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:23:54.091952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:23:54.092871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:23:54.092871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:23:54.092871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:23:54.092871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:23:54.092871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:23:54.092871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:23:54.092871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:23:54.092871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:23:54.092871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:23:54.092871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:23:54.092871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:23:54.092871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:23:54.092871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:23:54.092871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:23:54.092871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:23:54.092871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:23:54.093632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:23:54.093632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:23:54.093632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:23:54.093632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:23:54.093632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:23:54.093632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:23:54.093632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:23:54.093632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:23:54.093632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:23:54.093632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:23:54.093632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:23:54.093632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:23:54.093632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 58/110] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:23:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:23:54.094253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4067933-a341-4762-8195-c69c4ba2d146 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:54.119731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 53/111] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jun 6 10:23:53 2022] POST /identity/v3/projects => generated 302 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:23:54.128880 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c960c-1bca-4d75-a929-788fd5727240 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:54.231731 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4067933-a341-4762-8195-c69c4ba2d146 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:54.270775 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf55be-d9c5-4784-a613-1abf17584cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:54.271401 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf55be-d9c5-4784-a613-1abf17584cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:54.271981 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf55be-d9c5-4784-a613-1abf17584cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:54.272417 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf55be-d9c5-4784-a613-1abf17584cb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:54.273673 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c960c-1bca-4d75-a929-788fd5727240 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:54.310516 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:54.322495 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3201be-0863-4133-ae41-2212e4ac8ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:54.323647 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3201be-0863-4133-ae41-2212e4ac8ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:54.324228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3201be-0863-4133-ae41-2212e4ac8ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:54.324508 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3201be-0863-4133-ae41-2212e4ac8ae0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:54.343362 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3201be-0863-4133-ae41-2212e4ac8ae0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:23:54.345632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:54.353547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-7b3201be-0863-4133-ae41-2212e4ac8ae0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:23:54.353547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:23:54.353547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:23:54.353547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:23:54.353547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:23:54.353547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:23:54.353547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:23:54.353547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:23:54.353547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:23:54.353547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:23:54.353547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:23:54.353547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:23:54.353547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:23:54.353547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:23:54.353547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:23:54.353547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:23:54.354437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:23:54.354437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:23:54.354437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:23:54.354437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:23:54.354437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:23:54.354437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:23:54.354437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:23:54.354437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:23:54.354437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:23:54.354437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:23:54.354437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:23:54.354437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:23:54.354437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:23:54.354437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:23:54.354437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:23:54.354437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:23:54.355176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:23:54.355176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:23:54.355176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:23:54.355176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:23:54.355176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:23:54.355176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:23:54.355176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:23:54.355176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:23:54.355176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:23:54.355176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:23:54.355176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:23:54.355176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:23:54.356678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 54/112] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:23:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:23:54.357231 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79cf61-2ca1-4dd5-82f0-360b195ff8f7 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:54.357523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79cf61-2ca1-4dd5-82f0-360b195ff8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:54.357808 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79cf61-2ca1-4dd5-82f0-360b195ff8f7 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:54.363166 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 55/113] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:23:54.363166 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79cf61-2ca1-4dd5-82f0-360b195ff8f7 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:54.376845 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf55be-d9c5-4784-a613-1abf17584cb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:54.378660 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf55be-d9c5-4784-a613-1abf17584cb7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:54.491453 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 59/114] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:23:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:23:54.492637 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf55be-d9c5-4784-a613-1abf17584cb7 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:54.498943 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79cf61-2ca1-4dd5-82f0-360b195ff8f7 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:54.538470 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761a519-d934-407a-a5bb-7bbf06ccdb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:54.539059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761a519-d934-407a-a5bb-7bbf06ccdb6e None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:54.539343 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761a519-d934-407a-a5bb-7bbf06ccdb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:54.539615 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761a519-d934-407a-a5bb-7bbf06ccdb6e None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:54.540381 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a761a519-d934-407a-a5bb-7bbf06ccdb6e None admin] RBAC: Adding query filter params (name=member) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:54.540780 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a761a519-d934-407a-a5bb-7bbf06ccdb6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:54.542055 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a761a519-d934-407a-a5bb-7bbf06ccdb6e None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:54.551344 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 56/115] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:23:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:54.552688 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755cef9-9532-4a6f-a785-0baeb668a35a None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:54.553674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755cef9-9532-4a6f-a785-0baeb668a35a None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:54.554022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755cef9-9532-4a6f-a785-0baeb668a35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:54.571249 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-c755cef9-9532-4a6f-a785-0baeb668a35a None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:54.572497 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-c755cef9-9532-4a6f-a785-0baeb668a35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:54.585993 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:54.602518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf55be-d9c5-4784-a613-1abf17584cb7 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:54.621858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:54.640423 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c379de9b-ff6f-4eb6-9ca0-c89870919c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:54.641208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379de9b-ff6f-4eb6-9ca0-c89870919c78 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:54.641512 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379de9b-ff6f-4eb6-9ca0-c89870919c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:54.641766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379de9b-ff6f-4eb6-9ca0-c89870919c78 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:54.642516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c379de9b-ff6f-4eb6-9ca0-c89870919c78 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:54.642821 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c379de9b-ff6f-4eb6-9ca0-c89870919c78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:54.644241 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c379de9b-ff6f-4eb6-9ca0-c89870919c78 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:54.650228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 57/116] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:23:54.651947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb10fd-0e98-446a-b3ae-a818e3897da9 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:54.651947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb10fd-0e98-446a-b3ae-a818e3897da9 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:54.652330 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb10fd-0e98-446a-b3ae-a818e3897da9 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:54.657821 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 58/117] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:23:54.658420 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb10fd-0e98-446a-b3ae-a818e3897da9 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:54.665846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 60/118] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:23:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:54.670610 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c379de9b-ff6f-4eb6-9ca0-c89870919c78 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:54.770632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb10fd-0e98-446a-b3ae-a818e3897da9 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:54.793182 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c379de9b-ff6f-4eb6-9ca0-c89870919c78 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:54.807655 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8e8d8-f3fe-4965-b957-df190747b762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:54.808557 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8e8d8-f3fe-4965-b957-df190747b762 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:54.808840 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8e8d8-f3fe-4965-b957-df190747b762 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:54.809108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8e8d8-f3fe-4965-b957-df190747b762 None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:54.810275 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f8e8d8-f3fe-4965-b957-df190747b762 None admin] RBAC: Authorizing `identity:get_project(project_id=0efa83f60031447ab87be4f6f1af6165)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:54.811730 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f8e8d8-f3fe-4965-b957-df190747b762 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:54.813608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 59/119] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:23:54 2022] GET /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:54.817767 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6209c1d-68fa-4a41-a57e-2c7e4496b881 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:54.818592 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6209c1d-68fa-4a41-a57e-2c7e4496b881 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:54.818964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6209c1d-68fa-4a41-a57e-2c7e4496b881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:54.828112 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c87e4-230d-48ed-ad10-48cf98f246a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:54.829040 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c87e4-230d-48ed-ad10-48cf98f246a6 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:54.829323 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c87e4-230d-48ed-ad10-48cf98f246a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:54.829599 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c87e4-230d-48ed-ad10-48cf98f246a6 None admin] PATH_INFO: `/v3/users/709a0882f46a4bcaa3f47007f20cfaa1` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:54.830943 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c87e4-230d-48ed-ad10-48cf98f246a6 None admin] RBAC: Authorizing `identity:get_user(user_id=709a0882f46a4bcaa3f47007f20cfaa1)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:54.832503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c87e4-230d-48ed-ad10-48cf98f246a6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:54.840037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 61/120] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:23:54 2022] GET /identity/v3/users/709a0882f46a4bcaa3f47007f20cfaa1 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:54.840037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c87e4-230d-48ed-ad10-48cf98f246a6 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:54.841032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-b6209c1d-68fa-4a41-a57e-2c7e4496b881 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:54.842670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-b6209c1d-68fa-4a41-a57e-2c7e4496b881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:54.863038 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:54.892788 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:54.921090 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 60/121] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:23:54.923987 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6209c1d-68fa-4a41-a57e-2c7e4496b881 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:54.972488 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c87e4-230d-48ed-ad10-48cf98f246a6 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:55.008649 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbc05b-3c4e-4f55-aa77-35ae33c20ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:55.009530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbc05b-3c4e-4f55-aa77-35ae33c20ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:55.009813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbc05b-3c4e-4f55-aa77-35ae33c20ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:55.010208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbc05b-3c4e-4f55-aa77-35ae33c20ad9 None admin] PATH_INFO: `/v3/users/709a0882f46a4bcaa3f47007f20cfaa1` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:55.011295 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccbc05b-3c4e-4f55-aa77-35ae33c20ad9 None admin] RBAC: Authorizing `identity:get_user(user_id=709a0882f46a4bcaa3f47007f20cfaa1)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:55.013132 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccbc05b-3c4e-4f55-aa77-35ae33c20ad9 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:55.014628 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 62/122] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:23:54 2022] GET /identity/v3/users/709a0882f46a4bcaa3f47007f20cfaa1 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:55.019508 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbc05b-3c4e-4f55-aa77-35ae33c20ad9 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:55.026444 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6209c1d-68fa-4a41-a57e-2c7e4496b881 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:55.061723 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8bc35-e95f-4e5b-9d63-2a1765c28e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:55.062444 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8bc35-e95f-4e5b-9d63-2a1765c28e03 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:55.062728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8bc35-e95f-4e5b-9d63-2a1765c28e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:55.063004 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8bc35-e95f-4e5b-9d63-2a1765c28e03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:55.075179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8bc35-e95f-4e5b-9d63-2a1765c28e03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:23:55.083179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-7db8bc35-e95f-4e5b-9d63-2a1765c28e03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:23:55.083179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:23:55.083179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:23:55.083179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:23:55.083179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:23:55.083179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:23:55.083179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:23:55.083179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:23:55.083179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:23:55.083179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:23:55.083179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:23:55.083179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:23:55.083179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:23:55.083179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:23:55.083179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:23:55.083179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:23:55.084178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:23:55.084178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:23:55.084178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:23:55.084178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:23:55.084178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:23:55.084178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:23:55.084178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:23:55.084178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:23:55.084178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:23:55.084178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:23:55.084178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:23:55.084178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:23:55.084178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:23:55.084178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:23:55.084178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:23:55.084178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:23:55.085480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:23:55.085480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:23:55.085480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:23:55.085480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:23:55.085480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:23:55.085480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:23:55.085480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:23:55.085480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:23:55.085480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:23:55.085480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:23:55.085480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:23:55.085480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:23:55.085480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 61/123] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:23:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:23:55.086155 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8bc35-e95f-4e5b-9d63-2a1765c28e03 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:55.121191 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:55.126384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:55.142667 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbc05b-3c4e-4f55-aa77-35ae33c20ad9 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:55.180175 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60293cb-15d4-4257-a083-52572debcef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:55.181129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60293cb-15d4-4257-a083-52572debcef3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:55.181417 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60293cb-15d4-4257-a083-52572debcef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:55.181697 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60293cb-15d4-4257-a083-52572debcef3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:55.182528 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60293cb-15d4-4257-a083-52572debcef3 None admin] RBAC: Adding query filter params (role.id=803bb32940ee4c78ab77b3d63cd7ac0f, scope.project.id=0efa83f60031447ab87be4f6f1af6165, user.id=d518e1cdf98a415ebb623b7286bc4086) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:55.182842 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60293cb-15d4-4257-a083-52572debcef3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:55.184737 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60293cb-15d4-4257-a083-52572debcef3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:55.190515 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 63/124] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:23:55 2022] GET /identity/v3/role_assignments?user.id=d518e1cdf98a415ebb623b7286bc4086&scope.project.id=0efa83f60031447ab87be4f6f1af6165&role.id=803bb32940ee4c78ab77b3d63cd7ac0f => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:55.191290 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60293cb-15d4-4257-a083-52572debcef3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:55.202343 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8bc35-e95f-4e5b-9d63-2a1765c28e03 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:55.260851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aaeedb-15a2-44b1-8b3e-b5a046cb7111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:55.262982 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaeedb-15a2-44b1-8b3e-b5a046cb7111 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:55.263518 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaeedb-15a2-44b1-8b3e-b5a046cb7111 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:55.264259 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaeedb-15a2-44b1-8b3e-b5a046cb7111 None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:55.266659 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aaeedb-15a2-44b1-8b3e-b5a046cb7111 None admin] RBAC: Authorizing `identity:get_project(project_id=0efa83f60031447ab87be4f6f1af6165)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:55.268782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aaeedb-15a2-44b1-8b3e-b5a046cb7111 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:55.271924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 62/125] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:23:55 2022] GET /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:55.273022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aaeedb-15a2-44b1-8b3e-b5a046cb7111 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:55.324249 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60293cb-15d4-4257-a083-52572debcef3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:55.362678 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aab91e-9627-45aa-9d0c-15dd0318929a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:55.363745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aab91e-9627-45aa-9d0c-15dd0318929a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:55.363745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aab91e-9627-45aa-9d0c-15dd0318929a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:55.364078 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aab91e-9627-45aa-9d0c-15dd0318929a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:55.377146 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aab91e-9627-45aa-9d0c-15dd0318929a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:23:55.385598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-64aab91e-9627-45aa-9d0c-15dd0318929a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:23:55.385598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:23:55.385598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:23:55.385598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:23:55.385598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:23:55.385598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:23:55.385598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:23:55.385598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:23:55.385598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:23:55.385598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:23:55.385598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:23:55.385598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:23:55.385598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:23:55.385598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:23:55.385598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:23:55.385598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:23:55.386540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:23:55.386540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:23:55.386540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:23:55.386540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:23:55.386540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:23:55.386540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:23:55.386540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:23:55.386540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:23:55.386540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:23:55.386540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:23:55.386540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:23:55.386540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:23:55.386540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:23:55.386540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:23:55.386540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:23:55.386540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:23:55.387331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:23:55.387331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:23:55.387331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:23:55.387331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:23:55.387331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:23:55.387331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:23:55.387331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:23:55.387331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:23:55.387331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:23:55.387331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:23:55.387331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:23:55.387331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:23:55.387331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 64/126] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:23:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:23:55.393012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aab91e-9627-45aa-9d0c-15dd0318929a None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:55.404074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:55.419341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aaeedb-15a2-44b1-8b3e-b5a046cb7111 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:55.423437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:55.464678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b90c59-a77a-49f1-9a9a-44da2bb9a558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:55.465663 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b90c59-a77a-49f1-9a9a-44da2bb9a558 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:55.465949 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b90c59-a77a-49f1-9a9a-44da2bb9a558 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:55.466227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b90c59-a77a-49f1-9a9a-44da2bb9a558 None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:55.467930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b90c59-a77a-49f1-9a9a-44da2bb9a558 None admin] RBAC: Authorizing `identity:get_project(project_id=0efa83f60031447ab87be4f6f1af6165)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:55.469725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b90c59-a77a-49f1-9a9a-44da2bb9a558 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:55.477645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 63/127] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:23:55 2022] GET /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:55.477645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b90c59-a77a-49f1-9a9a-44da2bb9a558 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:55.531413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aab91e-9627-45aa-9d0c-15dd0318929a None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:55.577357 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18214bae-1e84-473f-8e20-e5b96c1aebcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:55.578531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18214bae-1e84-473f-8e20-e5b96c1aebcf None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:55.578969 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18214bae-1e84-473f-8e20-e5b96c1aebcf None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:55.579947 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18214bae-1e84-473f-8e20-e5b96c1aebcf None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:55.581167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18214bae-1e84-473f-8e20-e5b96c1aebcf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:55.581878 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18214bae-1e84-473f-8e20-e5b96c1aebcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:55.583841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18214bae-1e84-473f-8e20-e5b96c1aebcf None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:55.596794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 65/128] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:23:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:55.597885 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18214bae-1e84-473f-8e20-e5b96c1aebcf None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:55.607321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b90c59-a77a-49f1-9a9a-44da2bb9a558 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:55.650612 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f648d5-7ab3-45ac-a0c3-ba3eb0d4a814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:55.651734 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f648d5-7ab3-45ac-a0c3-ba3eb0d4a814 None admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:55.652155 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f648d5-7ab3-45ac-a0c3-ba3eb0d4a814 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:55.652589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f648d5-7ab3-45ac-a0c3-ba3eb0d4a814 None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165/users/709a0882f46a4bcaa3f47007f20cfaa1/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:55.655204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f648d5-7ab3-45ac-a0c3-ba3eb0d4a814 None admin] RBAC: Authorizing `identity:create_grant(project_id=0efa83f60031447ab87be4f6f1af6165, user_id=709a0882f46a4bcaa3f47007f20cfaa1, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:55.657290 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f648d5-7ab3-45ac-a0c3-ba3eb0d4a814 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:55.679769 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 64/129] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:23:55 2022] PUT /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165/users/709a0882f46a4bcaa3f47007f20cfaa1/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:23:55.680993 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f648d5-7ab3-45ac-a0c3-ba3eb0d4a814 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:55.688517 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:55.724167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:55.744593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18214bae-1e84-473f-8e20-e5b96c1aebcf None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:55.845358 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624fde2-e778-4ec9-92ab-f49dce22c9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:55.845968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624fde2-e778-4ec9-92ab-f49dce22c9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:55.846261 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624fde2-e778-4ec9-92ab-f49dce22c9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:55.846539 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624fde2-e778-4ec9-92ab-f49dce22c9e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:55.847314 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624fde2-e778-4ec9-92ab-f49dce22c9e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:55.847625 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624fde2-e778-4ec9-92ab-f49dce22c9e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:55.849164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624fde2-e778-4ec9-92ab-f49dce22c9e7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:55.853678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f648d5-7ab3-45ac-a0c3-ba3eb0d4a814 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:55.871891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 66/130] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:23:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:55.876596 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624fde2-e778-4ec9-92ab-f49dce22c9e7 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:55.905147 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ecad3-abf9-4512-b226-a109517c18ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:55.906598 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ecad3-abf9-4512-b226-a109517c18ff None admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:55.907097 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ecad3-abf9-4512-b226-a109517c18ff None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:55.907381 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ecad3-abf9-4512-b226-a109517c18ff None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165/users/709a0882f46a4bcaa3f47007f20cfaa1/roles/f214f8dbc3354cfc81fa2c2f116980e0` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:55.909720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ecad3-abf9-4512-b226-a109517c18ff None admin] RBAC: Authorizing `identity:create_grant(project_id=0efa83f60031447ab87be4f6f1af6165, user_id=709a0882f46a4bcaa3f47007f20cfaa1, role_id=f214f8dbc3354cfc81fa2c2f116980e0)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:55.911396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ecad3-abf9-4512-b226-a109517c18ff None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:55.929701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 65/131] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:23:55 2022] PUT /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165/users/709a0882f46a4bcaa3f47007f20cfaa1/roles/f214f8dbc3354cfc81fa2c2f116980e0 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:23:55.930598 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ecad3-abf9-4512-b226-a109517c18ff None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:56.029372 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624fde2-e778-4ec9-92ab-f49dce22c9e7 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:56.063530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:56.071968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df366b5-7fc3-4d45-85ed-3157b53f27f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:56.072874 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df366b5-7fc3-4d45-85ed-3157b53f27f0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:56.073161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df366b5-7fc3-4d45-85ed-3157b53f27f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:56.073437 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df366b5-7fc3-4d45-85ed-3157b53f27f0 None admin] PATH_INFO: `/v3/users/dc953d765b8048419427028bb5821273` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:56.074685 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df366b5-7fc3-4d45-85ed-3157b53f27f0 None admin] RBAC: Authorizing `identity:get_user(user_id=dc953d765b8048419427028bb5821273)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:56.076162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:56.076601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df366b5-7fc3-4d45-85ed-3157b53f27f0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:56.082448 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 67/132] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:23:55 2022] GET /identity/v3/users/dc953d765b8048419427028bb5821273 => generated 247 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:56.083931 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df366b5-7fc3-4d45-85ed-3157b53f27f0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:56.088254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ecad3-abf9-4512-b226-a109517c18ff None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:56.135069 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a5052-2664-4720-89fe-b91dac0f32b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:56.135972 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a5052-2664-4720-89fe-b91dac0f32b3 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:56.136255 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a5052-2664-4720-89fe-b91dac0f32b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:56.136532 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a5052-2664-4720-89fe-b91dac0f32b3 None admin] PATH_INFO: `/v3/users/709a0882f46a4bcaa3f47007f20cfaa1` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:56.137741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a5052-2664-4720-89fe-b91dac0f32b3 None admin] RBAC: Authorizing `identity:get_user(user_id=709a0882f46a4bcaa3f47007f20cfaa1)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:56.139175 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a5052-2664-4720-89fe-b91dac0f32b3 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:56.140883 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 66/133] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:23:55 2022] GET /identity/v3/users/709a0882f46a4bcaa3f47007f20cfaa1 => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:56.147285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a5052-2664-4720-89fe-b91dac0f32b3 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:56.221799 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df366b5-7fc3-4d45-85ed-3157b53f27f0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:56.267971 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae9f04-f4be-40e7-843e-d01886c7da52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:56.269168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae9f04-f4be-40e7-843e-d01886c7da52 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:56.269698 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae9f04-f4be-40e7-843e-d01886c7da52 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:56.270117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae9f04-f4be-40e7-843e-d01886c7da52 None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:56.271527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae9f04-f4be-40e7-843e-d01886c7da52 None admin] RBAC: Authorizing `identity:get_project(project_id=0efa83f60031447ab87be4f6f1af6165)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:56.274288 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae9f04-f4be-40e7-843e-d01886c7da52 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:56.276271 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 68/134] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:23:56 2022] GET /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:56.282401 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae9f04-f4be-40e7-843e-d01886c7da52 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:56.294193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a5052-2664-4720-89fe-b91dac0f32b3 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:56.332300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466a4c0-7c17-47c0-a2ff-622ac6cf4716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:56.333271 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466a4c0-7c17-47c0-a2ff-622ac6cf4716 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:56.333563 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466a4c0-7c17-47c0-a2ff-622ac6cf4716 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:56.333842 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466a4c0-7c17-47c0-a2ff-622ac6cf4716 None admin] PATH_INFO: `/v3/projects/09ed03f605ac4be88166564d72a5e9e3` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:56.336178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f466a4c0-7c17-47c0-a2ff-622ac6cf4716 None admin] RBAC: Authorizing `identity:get_project(project_id=09ed03f605ac4be88166564d72a5e9e3)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:56.337760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f466a4c0-7c17-47c0-a2ff-622ac6cf4716 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:56.339933 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 67/135] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:23:56 2022] GET /identity/v3/projects/09ed03f605ac4be88166564d72a5e9e3 => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:23:56.345384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466a4c0-7c17-47c0-a2ff-622ac6cf4716 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:56.366950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:56.406390 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:56.434333 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae9f04-f4be-40e7-843e-d01886c7da52 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:56.510595 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92bff19-9211-4eaa-8e6d-496bc802f40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:56.511521 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bff19-9211-4eaa-8e6d-496bc802f40e None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:56.511978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bff19-9211-4eaa-8e6d-496bc802f40e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:56.512276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bff19-9211-4eaa-8e6d-496bc802f40e None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165/users/dc953d765b8048419427028bb5821273/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:56.517732 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92bff19-9211-4eaa-8e6d-496bc802f40e None admin] RBAC: Authorizing `identity:create_grant(project_id=0efa83f60031447ab87be4f6f1af6165, user_id=dc953d765b8048419427028bb5821273, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:56.519202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92bff19-9211-4eaa-8e6d-496bc802f40e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:56.548012 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466a4c0-7c17-47c0-a2ff-622ac6cf4716 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:56.551638 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 69/136] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:23:56 2022] PUT /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165/users/dc953d765b8048419427028bb5821273/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:23:56.610415 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774c5d8-af82-46a2-98ce-ad6062eb0cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:56.611337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774c5d8-af82-46a2-98ce-ad6062eb0cdb None admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:56.611618 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774c5d8-af82-46a2-98ce-ad6062eb0cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:56.611919 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774c5d8-af82-46a2-98ce-ad6062eb0cdb None admin] PATH_INFO: `/v3/projects/09ed03f605ac4be88166564d72a5e9e3/users/709a0882f46a4bcaa3f47007f20cfaa1/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:56.614226 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774c5d8-af82-46a2-98ce-ad6062eb0cdb None admin] RBAC: Authorizing `identity:create_grant(project_id=09ed03f605ac4be88166564d72a5e9e3, user_id=709a0882f46a4bcaa3f47007f20cfaa1, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:56.616059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774c5d8-af82-46a2-98ce-ad6062eb0cdb None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:56.641383 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 68/137] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:23:56 2022] PUT /identity/v3/projects/09ed03f605ac4be88166564d72a5e9e3/users/709a0882f46a4bcaa3f47007f20cfaa1/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:23:56.931298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760a8f2-ef35-427d-8c89-11454aa0617e None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:56.931657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760a8f2-ef35-427d-8c89-11454aa0617e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:56.931764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760a8f2-ef35-427d-8c89-11454aa0617e None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:56.933314 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 70/138] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:23:56.938829 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb02db-220e-4c3c-8f65-caadac981298 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:56.939123 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb02db-220e-4c3c-8f65-caadac981298 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:56.939406 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb02db-220e-4c3c-8f65-caadac981298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:56.961953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-6eeb02db-220e-4c3c-8f65-caadac981298 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:56.963165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-6eeb02db-220e-4c3c-8f65-caadac981298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:57.042085 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 69/139] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:23:57.067596 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760a8f2-ef35-427d-8c89-11454aa0617e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:57.178352 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760a8f2-ef35-427d-8c89-11454aa0617e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:57.204079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:57.214535 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1026d-f0d9-4ddc-947d-743b39119ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:57.215340 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1026d-f0d9-4ddc-947d-743b39119ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:57.215618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1026d-f0d9-4ddc-947d-743b39119ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:57.215921 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1026d-f0d9-4ddc-947d-743b39119ee6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:57.217186 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1026d-f0d9-4ddc-947d-743b39119ee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:57.219916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1026d-f0d9-4ddc-947d-743b39119ee6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:57.220285 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 71/140] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:23:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:57.227026 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeb02db-220e-4c3c-8f65-caadac981298 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:57.246681 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:57.355118 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeb02db-220e-4c3c-8f65-caadac981298 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:57.395232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68156a58-60e8-4016-a81d-e1d217eebfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:57.395941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68156a58-60e8-4016-a81d-e1d217eebfda None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:57.396238 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68156a58-60e8-4016-a81d-e1d217eebfda None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:57.396613 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68156a58-60e8-4016-a81d-e1d217eebfda None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:57.398401 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68156a58-60e8-4016-a81d-e1d217eebfda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:57.400167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68156a58-60e8-4016-a81d-e1d217eebfda None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:57.431740 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 70/141] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jun 6 10:23:57 2022] POST /identity/v3/users => generated 283 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:23:58.745896 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d27f6-8e10-4e29-98c1-f4bd0c64678c None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:58.746428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d27f6-8e10-4e29-98c1-f4bd0c64678c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:58.746428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d27f6-8e10-4e29-98c1-f4bd0c64678c None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:58.748183 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 72/142] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:23:58.753200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a673a-8f3d-442c-a8eb-0f07e7997bfe None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:58.753631 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a673a-8f3d-442c-a8eb-0f07e7997bfe None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:58.753810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a673a-8f3d-442c-a8eb-0f07e7997bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:58.776949 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-f64a673a-8f3d-442c-a8eb-0f07e7997bfe None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:58.778171 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-f64a673a-8f3d-442c-a8eb-0f07e7997bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:58.869857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 71/143] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:23:58.905557 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d27f6-8e10-4e29-98c1-f4bd0c64678c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:59.021284 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d27f6-8e10-4e29-98c1-f4bd0c64678c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:59.062260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d071d-c92a-4e82-b024-49b633888725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:59.063060 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d071d-c92a-4e82-b024-49b633888725 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:59.063339 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d071d-c92a-4e82-b024-49b633888725 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:59.063608 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d071d-c92a-4e82-b024-49b633888725 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:59.079753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d071d-c92a-4e82-b024-49b633888725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:23:59.088788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-af5d071d-c92a-4e82-b024-49b633888725 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:23:59.088788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:23:59.088788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:23:59.088788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:23:59.088788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:23:59.088788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:23:59.088788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:23:59.088788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:23:59.088788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:23:59.088788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:23:59.088788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:23:59.088788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:23:59.088788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:23:59.088788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:23:59.088788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:23:59.088788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:23:59.089763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:23:59.089763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:23:59.089763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:23:59.089763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:23:59.089763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:23:59.089763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:23:59.089763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:23:59.089763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:23:59.089763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:23:59.089763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:23:59.089763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:23:59.089763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:23:59.089763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:23:59.089763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:23:59.089763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:23:59.089763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:23:59.090520 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:23:59.090520 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:23:59.090520 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:23:59.090520 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:23:59.090520 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:23:59.090520 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:23:59.090520 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:23:59.090520 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:23:59.090520 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:23:59.090520 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:23:59.090520 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:23:59.090520 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:23:59.090520 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 73/144] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:23:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:23:59.095687 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a673a-8f3d-442c-a8eb-0f07e7997bfe None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:59.098932 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:59.134002 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f4a41-ef76-4677-8fdc-68572ca83e52 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:59.134320 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f4a41-ef76-4677-8fdc-68572ca83e52 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:59.134547 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f4a41-ef76-4677-8fdc-68572ca83e52 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:59.136092 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 74/145] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:23:59.141250 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91b689-cb21-43fb-a5eb-9ffebd5deb44 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:59.141536 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91b689-cb21-43fb-a5eb-9ffebd5deb44 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:59.141816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91b689-cb21-43fb-a5eb-9ffebd5deb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:59.172939 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ed91b689-cb21-43fb-a5eb-9ffebd5deb44 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:23:59.177091 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ed91b689-cb21-43fb-a5eb-9ffebd5deb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:23:59.271114 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a673a-8f3d-442c-a8eb-0f07e7997bfe None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:59.294088 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:59.315252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a456a4-2d24-4982-a57c-e138b89d0672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:59.315970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a456a4-2d24-4982-a57c-e138b89d0672 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:59.316256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a456a4-2d24-4982-a57c-e138b89d0672 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:59.316535 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a456a4-2d24-4982-a57c-e138b89d0672 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:59.317444 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a456a4-2d24-4982-a57c-e138b89d0672 None admin] RBAC: Adding query filter params (name=member) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:59.317755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a456a4-2d24-4982-a57c-e138b89d0672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:59.320338 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a456a4-2d24-4982-a57c-e138b89d0672 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:59.324334 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 75/146] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 10:23:59.329823 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 72/147] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:23:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:23:59.335745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91b689-cb21-43fb-a5eb-9ffebd5deb44 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:59.358536 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a456a4-2d24-4982-a57c-e138b89d0672 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:59.457280 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91b689-cb21-43fb-a5eb-9ffebd5deb44 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:59.466002 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:59.500593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a250de07-b21c-4c5f-a2ad-cb300ab2497a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:59.501607 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250de07-b21c-4c5f-a2ad-cb300ab2497a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:59.501888 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250de07-b21c-4c5f-a2ad-cb300ab2497a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:59.502161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250de07-b21c-4c5f-a2ad-cb300ab2497a None admin] PATH_INFO: `/v3/users/709a0882f46a4bcaa3f47007f20cfaa1` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:59.503607 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a250de07-b21c-4c5f-a2ad-cb300ab2497a None admin] RBAC: Authorizing `identity:get_user(user_id=709a0882f46a4bcaa3f47007f20cfaa1)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:59.505123 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a250de07-b21c-4c5f-a2ad-cb300ab2497a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:59.507284 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 76/148] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:23:59 2022] GET /identity/v3/users/709a0882f46a4bcaa3f47007f20cfaa1 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:59.509132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a456a4-2d24-4982-a57c-e138b89d0672 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:59.512559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a250de07-b21c-4c5f-a2ad-cb300ab2497a None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:59.550598 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfae06-2d3d-4dc1-af91-7500846e2ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:59.551419 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfae06-2d3d-4dc1-af91-7500846e2ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:59.551701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfae06-2d3d-4dc1-af91-7500846e2ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:59.552023 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfae06-2d3d-4dc1-af91-7500846e2ca2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:59.564988 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bfae06-2d3d-4dc1-af91-7500846e2ca2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:23:59.576320 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:59.578805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-31bfae06-2d3d-4dc1-af91-7500846e2ca2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 10:23:59.578805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:23:59.578805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:23:59.578805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:23:59.578805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:23:59.578805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:23:59.578805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:23:59.578805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:23:59.578805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:23:59.578805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:23:59.578805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:23:59.578805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:23:59.578805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:23:59.578805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:23:59.578805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:23:59.578805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:23:59.582728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:23:59.582728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:23:59.582728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:23:59.582728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:23:59.582728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:23:59.582728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:23:59.582728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:23:59.582728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:23:59.582728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:23:59.582728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:23:59.582728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:23:59.582728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:23:59.582728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:23:59.582728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:23:59.582728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:23:59.582728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:23:59.584881 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:23:59.584881 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:23:59.584881 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:23:59.584881 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:23:59.584881 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:23:59.584881 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:23:59.584881 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:23:59.584881 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:23:59.584881 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:23:59.584881 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:23:59.584881 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 10:23:59.584881 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:23:59.584881 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 73/149] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:23:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:23:59.586461 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfae06-2d3d-4dc1-af91-7500846e2ca2 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:59.625337 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a250de07-b21c-4c5f-a2ad-cb300ab2497a None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:59.679955 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e64ab-932a-40e8-8f4e-f7f06264c9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:59.680617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e64ab-932a-40e8-8f4e-f7f06264c9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:59.680900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e64ab-932a-40e8-8f4e-f7f06264c9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:59.681177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e64ab-932a-40e8-8f4e-f7f06264c9f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:59.712889 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfae06-2d3d-4dc1-af91-7500846e2ca2 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:23:59.742021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:59.751684 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a43ce-6290-45ac-aec1-e21d66e1d24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:23:59.752789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a43ce-6290-45ac-aec1-e21d66e1d24c None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:23:59.753290 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a43ce-6290-45ac-aec1-e21d66e1d24c None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:23:59.753908 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a43ce-6290-45ac-aec1-e21d66e1d24c None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:23:59.754408 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a43ce-6290-45ac-aec1-e21d66e1d24c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:23:59.754712 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a43ce-6290-45ac-aec1-e21d66e1d24c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:59.756703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a43ce-6290-45ac-aec1-e21d66e1d24c None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:59.767822 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 74/150] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:23:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:23:59.775890 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a43ce-6290-45ac-aec1-e21d66e1d24c None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:23:59.787467 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e64ab-932a-40e8-8f4e-f7f06264c9f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:23:59.789241 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e64ab-932a-40e8-8f4e-f7f06264c9f4 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:23:59.917079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:23:59.945992 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a43ce-6290-45ac-aec1-e21d66e1d24c None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:00.045502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c652c5-e99f-437b-b219-e36635b529be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:00.046395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c652c5-e99f-437b-b219-e36635b529be None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:00.046679 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c652c5-e99f-437b-b219-e36635b529be None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:00.046953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c652c5-e99f-437b-b219-e36635b529be None admin] PATH_INFO: `/v3/users/d518e1cdf98a415ebb623b7286bc4086` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:00.048735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 77/151] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:23:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:00.052555 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c652c5-e99f-437b-b219-e36635b529be None admin] RBAC: Authorizing `identity:get_user(user_id=d518e1cdf98a415ebb623b7286bc4086)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:00.054219 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e64ab-932a-40e8-8f4e-f7f06264c9f4 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:00.054536 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c652c5-e99f-437b-b219-e36635b529be None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:00.056442 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 75/152] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:23:59 2022] GET /identity/v3/users/d518e1cdf98a415ebb623b7286bc4086 => generated 289 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:00.061573 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c652c5-e99f-437b-b219-e36635b529be None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:00.202651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:00.275145 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c652c5-e99f-437b-b219-e36635b529be None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:00.280321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e64ab-932a-40e8-8f4e-f7f06264c9f4 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:00.347192 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91cbfd-0134-4dca-b874-eb9b8ae4ad5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:00.348107 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91cbfd-0134-4dca-b874-eb9b8ae4ad5c None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:00.348388 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91cbfd-0134-4dca-b874-eb9b8ae4ad5c None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:00.348660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91cbfd-0134-4dca-b874-eb9b8ae4ad5c None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:00.350010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91cbfd-0134-4dca-b874-eb9b8ae4ad5c None admin] RBAC: Authorizing `identity:get_project(project_id=0efa83f60031447ab87be4f6f1af6165)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:00.351489 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91cbfd-0134-4dca-b874-eb9b8ae4ad5c None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:00.361612 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 76/153] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:00 2022] GET /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165 => generated 298 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:00.362712 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9690c3-4137-48c1-ac7c-c8756a715342 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:00.363001 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9690c3-4137-48c1-ac7c-c8756a715342 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:00.363278 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9690c3-4137-48c1-ac7c-c8756a715342 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:00.364914 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 77/154] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:00.366237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac24adb-6f00-45ad-a859-37f80113413d None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:00.366425 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac24adb-6f00-45ad-a859-37f80113413d None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:00.366688 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac24adb-6f00-45ad-a859-37f80113413d None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:00.368067 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 78/155] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:00.368593 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac24adb-6f00-45ad-a859-37f80113413d None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:00.373140 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efbc24-5e45-4059-83bf-62e0b8d3b046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:00.374024 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efbc24-5e45-4059-83bf-62e0b8d3b046 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:00.374435 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efbc24-5e45-4059-83bf-62e0b8d3b046 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:00.374716 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efbc24-5e45-4059-83bf-62e0b8d3b046 None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:00.375957 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efbc24-5e45-4059-83bf-62e0b8d3b046 None admin] RBAC: Authorizing `identity:get_project(project_id=0efa83f60031447ab87be4f6f1af6165)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:00.377417 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efbc24-5e45-4059-83bf-62e0b8d3b046 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:00.379260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 78/156] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:00 2022] GET /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165 => generated 298 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:00.385231 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e6794-7c4a-4e5e-88a1-bf7dc10a693d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:00.385518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e6794-7c4a-4e5e-88a1-bf7dc10a693d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:00.385801 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e6794-7c4a-4e5e-88a1-bf7dc10a693d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:00.421060 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3e0e6794-7c4a-4e5e-88a1-bf7dc10a693d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:00.422444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3e0e6794-7c4a-4e5e-88a1-bf7dc10a693d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:00.451264 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:00.589235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 79/157] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:00.590472 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed0566-a957-430e-9e1e-9f5979886b65 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:00.590766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed0566-a957-430e-9e1e-9f5979886b65 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:00.591045 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed0566-a957-430e-9e1e-9f5979886b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:00.611517 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac24adb-6f00-45ad-a859-37f80113413d None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:00.628840 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-38ed0566-a957-430e-9e1e-9f5979886b65 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:00.630369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-38ed0566-a957-430e-9e1e-9f5979886b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:00.637756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:00.651880 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38d169-61bb-403f-b161-af7fc504d608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:00.651880 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38d169-61bb-403f-b161-af7fc504d608 None admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:00.651880 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38d169-61bb-403f-b161-af7fc504d608 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:00.651880 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38d169-61bb-403f-b161-af7fc504d608 None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165/users/d518e1cdf98a415ebb623b7286bc4086/roles/803bb32940ee4c78ab77b3d63cd7ac0f` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:00.653687 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38d169-61bb-403f-b161-af7fc504d608 None admin] RBAC: Authorizing `identity:create_grant(project_id=0efa83f60031447ab87be4f6f1af6165, user_id=d518e1cdf98a415ebb623b7286bc4086, role_id=803bb32940ee4c78ab77b3d63cd7ac0f)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:00.655172 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38d169-61bb-403f-b161-af7fc504d608 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:00.675988 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 79/158] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:24:00 2022] PUT /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165/users/d518e1cdf98a415ebb623b7286bc4086/roles/803bb32940ee4c78ab77b3d63cd7ac0f => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:24:00.676593 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38d169-61bb-403f-b161-af7fc504d608 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:00.781378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 80/159] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:00.782568 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20affcbe-4bdb-4c65-8522-e3238b2033e5 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:00.782855 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20affcbe-4bdb-4c65-8522-e3238b2033e5 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:00.783131 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20affcbe-4bdb-4c65-8522-e3238b2033e5 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:00.784772 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 81/160] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:00.785330 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20affcbe-4bdb-4c65-8522-e3238b2033e5 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:00.840294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38d169-61bb-403f-b161-af7fc504d608 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:00.880654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc071e-913e-46ab-9018-8288d01bff4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:00.882009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc071e-913e-46ab-9018-8288d01bff4f None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:00.882294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc071e-913e-46ab-9018-8288d01bff4f None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:00.882563 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc071e-913e-46ab-9018-8288d01bff4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:00.883502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc071e-913e-46ab-9018-8288d01bff4f None admin] RBAC: Adding query filter params (role.id=6bcb06bf4aa4442ebcfebeadf1af41de, scope.project.id=0efa83f60031447ab87be4f6f1af6165, user.id=709a0882f46a4bcaa3f47007f20cfaa1) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:00.883839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc071e-913e-46ab-9018-8288d01bff4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:00.886059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc071e-913e-46ab-9018-8288d01bff4f None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:00.893842 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 80/161] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:24:00 2022] GET /identity/v3/role_assignments?user.id=709a0882f46a4bcaa3f47007f20cfaa1&scope.project.id=0efa83f60031447ab87be4f6f1af6165&role.id=6bcb06bf4aa4442ebcfebeadf1af41de => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:00.894918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33869bab-50d0-4540-ad4f-08792b89cec4 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:00.895258 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33869bab-50d0-4540-ad4f-08792b89cec4 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:00.895476 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33869bab-50d0-4540-ad4f-08792b89cec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:00.940742 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-33869bab-50d0-4540-ad4f-08792b89cec4 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:00.942006 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-33869bab-50d0-4540-ad4f-08792b89cec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:01.012186 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:01.048168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20affcbe-4bdb-4c65-8522-e3238b2033e5 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:01.050042 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:01.104233 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7b62b-5c60-4012-9423-714ac6157a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:01.105149 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7b62b-5c60-4012-9423-714ac6157a41 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:01.105501 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7b62b-5c60-4012-9423-714ac6157a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:01.105849 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7b62b-5c60-4012-9423-714ac6157a41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:01.106833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 81/162] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:01.108070 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33869bab-50d0-4540-ad4f-08792b89cec4 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:01.122999 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7b62b-5c60-4012-9423-714ac6157a41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:01.138335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-29a7b62b-5c60-4012-9423-714ac6157a41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:24:01.138335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:01.138335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:01.138335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:01.138335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:01.138335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:01.138335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:01.138335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:01.138335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:01.138335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:01.138335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:01.138335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:01.138335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:01.138335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:01.138335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:01.138335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:01.139266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:01.139266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:01.139266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:01.139266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:01.139266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:01.139266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:01.139266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:01.139266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:01.139266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:01.139266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:01.139266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:01.139266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:01.139266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:01.139266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:01.139266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:01.139266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:01.140084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:01.140084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:01.140084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:01.140084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:01.140084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:01.140084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:01.140084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:01.140084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:01.140084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:01.140084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:01.140084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:24:01.140084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:01.140084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 82/163] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:24:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:01.148314 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7b62b-5c60-4012-9423-714ac6157a41 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:01.265993 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33869bab-50d0-4540-ad4f-08792b89cec4 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:01.326347 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7b62b-5c60-4012-9423-714ac6157a41 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:01.329866 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb236277-f084-4795-bd8d-de9b5d722c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:01.330912 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb236277-f084-4795-bd8d-de9b5d722c2e None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:01.331414 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb236277-f084-4795-bd8d-de9b5d722c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:01.331904 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb236277-f084-4795-bd8d-de9b5d722c2e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:01.349119 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb236277-f084-4795-bd8d-de9b5d722c2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:01.362534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-fb236277-f084-4795-bd8d-de9b5d722c2e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:01.362534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:01.362534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:01.362534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:01.362534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:01.362534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:01.362534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:01.362534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:01.362534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:01.362534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:01.362534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:01.362534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:01.362534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:01.362534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:01.362534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:01.362534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:01.363478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:01.363478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:01.363478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:01.363478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:01.363478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:01.363478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:01.363478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:01.363478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:01.363478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:01.363478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:01.363478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:01.363478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:01.363478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:01.363478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:01.363478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:01.363478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:01.368665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:01.368665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:01.368665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:01.368665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:01.368665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:01.368665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:01.368665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:01.368665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:01.368665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:01.368665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:01.368665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:01.368665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:01.371374 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 82/164] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:24:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:01.372770 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb236277-f084-4795-bd8d-de9b5d722c2e None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:01.382108 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b763bdb-129c-450e-b25a-adbd5ef855d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:01.382791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b763bdb-129c-450e-b25a-adbd5ef855d1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:01.383076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b763bdb-129c-450e-b25a-adbd5ef855d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:01.383350 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b763bdb-129c-450e-b25a-adbd5ef855d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:01.387828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b763bdb-129c-450e-b25a-adbd5ef855d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:01.387828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b763bdb-129c-450e-b25a-adbd5ef855d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:01.387828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b763bdb-129c-450e-b25a-adbd5ef855d1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:01.400591 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 83/165] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:01.401455 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b763bdb-129c-450e-b25a-adbd5ef855d1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:01.416166 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:01.448355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:01.522088 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b763bdb-129c-450e-b25a-adbd5ef855d1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:01.553743 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb236277-f084-4795-bd8d-de9b5d722c2e None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:01.560182 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819d616-8f6c-4a70-86a7-007cea2d0782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:01.561049 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819d616-8f6c-4a70-86a7-007cea2d0782 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:01.561489 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819d616-8f6c-4a70-86a7-007cea2d0782 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:01.561922 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819d616-8f6c-4a70-86a7-007cea2d0782 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:01.562800 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819d616-8f6c-4a70-86a7-007cea2d0782 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:01.563230 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819d616-8f6c-4a70-86a7-007cea2d0782 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:01.564774 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819d616-8f6c-4a70-86a7-007cea2d0782 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:01.576084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 84/166] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:24:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:01.577166 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819d616-8f6c-4a70-86a7-007cea2d0782 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:01.611151 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558657b-0e62-4783-85d3-0d191066769b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:01.612142 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558657b-0e62-4783-85d3-0d191066769b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:01.612373 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558657b-0e62-4783-85d3-0d191066769b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:01.612798 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558657b-0e62-4783-85d3-0d191066769b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:01.628608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558657b-0e62-4783-85d3-0d191066769b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:01.641372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-1558657b-0e62-4783-85d3-0d191066769b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:01.641372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:01.641372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:01.641372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:01.641372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:01.641372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:01.641372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:01.641372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:01.641372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:01.641372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:01.641372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:01.641372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:01.641372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:01.641372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:01.641372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:01.641372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:01.642307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:01.642307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:01.642307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:01.642307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:01.642307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:01.642307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:01.642307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:01.642307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:01.642307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:01.642307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:01.642307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:01.642307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:01.642307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:01.642307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:01.642307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:01.642307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:01.643110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:01.643110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:01.643110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:01.643110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:01.643110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:01.643110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:01.643110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:01.643110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:01.643110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:01.643110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:01.643110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:01.643110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:01.643110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 83/167] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:01.646304 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558657b-0e62-4783-85d3-0d191066769b None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:01.663456 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:01.698276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819d616-8f6c-4a70-86a7-007cea2d0782 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:01.734190 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bcc0e-f824-45a7-b2db-b062d0c928da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:01.735511 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bcc0e-f824-45a7-b2db-b062d0c928da None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:01.736056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bcc0e-f824-45a7-b2db-b062d0c928da None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:01.736578 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bcc0e-f824-45a7-b2db-b062d0c928da None admin] PATH_INFO: `/v3/users/dc953d765b8048419427028bb5821273` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:01.738077 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695bcc0e-f824-45a7-b2db-b062d0c928da None admin] RBAC: Authorizing `identity:get_user(user_id=dc953d765b8048419427028bb5821273)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:01.739874 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695bcc0e-f824-45a7-b2db-b062d0c928da None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:01.743037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 85/168] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:01 2022] GET /identity/v3/users/dc953d765b8048419427028bb5821273 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:01.744058 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bcc0e-f824-45a7-b2db-b062d0c928da None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:01.807725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:01.811485 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558657b-0e62-4783-85d3-0d191066769b None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:01.876485 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76917f6e-b056-4732-af84-6fe22d95550d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:01.877096 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76917f6e-b056-4732-af84-6fe22d95550d None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:01.877340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76917f6e-b056-4732-af84-6fe22d95550d None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:01.877500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76917f6e-b056-4732-af84-6fe22d95550d None admin] PATH_INFO: `/v3/users/709a0882f46a4bcaa3f47007f20cfaa1` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:01.878704 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76917f6e-b056-4732-af84-6fe22d95550d None admin] RBAC: Authorizing `identity:get_user(user_id=709a0882f46a4bcaa3f47007f20cfaa1)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:01.887265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76917f6e-b056-4732-af84-6fe22d95550d None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:01.887265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 84/169] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:01 2022] GET /identity/v3/users/709a0882f46a4bcaa3f47007f20cfaa1 => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:01.887265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76917f6e-b056-4732-af84-6fe22d95550d None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:01.894605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bcc0e-f824-45a7-b2db-b062d0c928da None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:01.944331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:01.945850 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d20f4-f072-424c-95bd-b333225d84b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:01.946926 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d20f4-f072-424c-95bd-b333225d84b7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:01.947515 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d20f4-f072-424c-95bd-b333225d84b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:01.948135 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d20f4-f072-424c-95bd-b333225d84b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:01.949913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d20f4-f072-424c-95bd-b333225d84b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:01.949913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d20f4-f072-424c-95bd-b333225d84b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:01.953536 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d20f4-f072-424c-95bd-b333225d84b7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:01.967510 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 86/170] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:24:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:01.968804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d20f4-f072-424c-95bd-b333225d84b7 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:02.014642 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76917f6e-b056-4732-af84-6fe22d95550d None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:02.055758 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f0495-4951-4a7c-ac7f-bdeab279103d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:02.055758 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f0495-4951-4a7c-ac7f-bdeab279103d None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:02.055758 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f0495-4951-4a7c-ac7f-bdeab279103d None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:02.055758 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f0495-4951-4a7c-ac7f-bdeab279103d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:02.098891 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:02.125223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f0495-4951-4a7c-ac7f-bdeab279103d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:02.126775 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f0495-4951-4a7c-ac7f-bdeab279103d None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:02.237406 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 85/171] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:02.238634 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f0495-4951-4a7c-ac7f-bdeab279103d None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:02.242766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d20f4-f072-424c-95bd-b333225d84b7 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:02.309592 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:02.360153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f0495-4951-4a7c-ac7f-bdeab279103d None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:02.424392 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c845b6-8f90-4f62-8432-f78d4abad168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:02.425384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c845b6-8f90-4f62-8432-f78d4abad168 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:02.425668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c845b6-8f90-4f62-8432-f78d4abad168 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:02.425940 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c845b6-8f90-4f62-8432-f78d4abad168 None admin] PATH_INFO: `/v3/users/709a0882f46a4bcaa3f47007f20cfaa1` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:02.427219 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c845b6-8f90-4f62-8432-f78d4abad168 None admin] RBAC: Authorizing `identity:get_user(user_id=709a0882f46a4bcaa3f47007f20cfaa1)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:02.429751 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c845b6-8f90-4f62-8432-f78d4abad168 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:02.432023 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 86/172] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:02 2022] GET /identity/v3/users/709a0882f46a4bcaa3f47007f20cfaa1 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:02.432966 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c845b6-8f90-4f62-8432-f78d4abad168 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:02.435042 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b96c67-eead-4779-9cf5-6b52ea1d2a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:02.435727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b96c67-eead-4779-9cf5-6b52ea1d2a6a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:02.436043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b96c67-eead-4779-9cf5-6b52ea1d2a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:02.436324 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b96c67-eead-4779-9cf5-6b52ea1d2a6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:02.601151 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c845b6-8f90-4f62-8432-f78d4abad168 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:02.620987 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:02.633419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b96c67-eead-4779-9cf5-6b52ea1d2a6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:02.636585 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b96c67-eead-4779-9cf5-6b52ea1d2a6a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:02.687810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d52107-df8a-4f3c-8521-69d705c223bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:02.687810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d52107-df8a-4f3c-8521-69d705c223bc None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:02.687810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d52107-df8a-4f3c-8521-69d705c223bc None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:02.687810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d52107-df8a-4f3c-8521-69d705c223bc None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:02.688225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d52107-df8a-4f3c-8521-69d705c223bc None admin] RBAC: Authorizing `identity:get_project(project_id=0efa83f60031447ab87be4f6f1af6165)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:02.688225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d52107-df8a-4f3c-8521-69d705c223bc None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:02.688358 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 87/173] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:02 2022] GET /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165 => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:02.689137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d52107-df8a-4f3c-8521-69d705c223bc None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:02.692049 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:02.819607 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d52107-df8a-4f3c-8521-69d705c223bc None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:02.829425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 87/174] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 861 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:02.830763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac79562-1f32-4dc9-9562-a44959bccdaa None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:02.831056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac79562-1f32-4dc9-9562-a44959bccdaa None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:02.831332 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac79562-1f32-4dc9-9562-a44959bccdaa None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:02.832644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 88/175] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:02.833348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac79562-1f32-4dc9-9562-a44959bccdaa None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:02.870243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db3dd8-e900-4a04-902e-6771a18b0c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:02.870892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db3dd8-e900-4a04-902e-6771a18b0c44 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:02.871290 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db3dd8-e900-4a04-902e-6771a18b0c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:02.871836 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db3dd8-e900-4a04-902e-6771a18b0c44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:02.948824 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db3dd8-e900-4a04-902e-6771a18b0c44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:02.950983 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db3dd8-e900-4a04-902e-6771a18b0c44 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:02.953176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:02.958238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:02.970212 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac79562-1f32-4dc9-9562-a44959bccdaa None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:03.011465 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc3b9c-26b2-4b83-b907-58bccffda7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:03.012398 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc3b9c-26b2-4b83-b907-58bccffda7fb None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:03.012900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc3b9c-26b2-4b83-b907-58bccffda7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:03.013319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc3b9c-26b2-4b83-b907-58bccffda7fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:03.014293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc3b9c-26b2-4b83-b907-58bccffda7fb None admin] RBAC: Adding query filter params (role.id=0f929204eb7b4ea8859338ed2d3ff75b, scope.project.id=0efa83f60031447ab87be4f6f1af6165, user.id=dc953d765b8048419427028bb5821273) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:03.014748 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc3b9c-26b2-4b83-b907-58bccffda7fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:03.016374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc3b9c-26b2-4b83-b907-58bccffda7fb None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:03.022635 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 89/176] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:24:02 2022] GET /identity/v3/role_assignments?user.id=dc953d765b8048419427028bb5821273&scope.project.id=0efa83f60031447ab87be4f6f1af6165&role.id=0f929204eb7b4ea8859338ed2d3ff75b => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:03.024398 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d4f23-c8d5-4490-a254-6616258a8325 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:03.024830 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d4f23-c8d5-4490-a254-6616258a8325 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:03.025249 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d4f23-c8d5-4490-a254-6616258a8325 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:03.026714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 90/177] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:03.027657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d4f23-c8d5-4490-a254-6616258a8325 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:03.084240 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 88/178] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:03.086172 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e083fb-773d-4de3-8303-6baf4c609b51 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:03.086608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e083fb-773d-4de3-8303-6baf4c609b51 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:03.086890 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e083fb-773d-4de3-8303-6baf4c609b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:03.114825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-12e083fb-773d-4de3-8303-6baf4c609b51 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:03.116958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-12e083fb-773d-4de3-8303-6baf4c609b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:03.165977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d4f23-c8d5-4490-a254-6616258a8325 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:03.208339 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45faf0a5-51da-4b65-bb64-30fec4b3b1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:03.209457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45faf0a5-51da-4b65-bb64-30fec4b3b1d6 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:03.210090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45faf0a5-51da-4b65-bb64-30fec4b3b1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:03.210428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45faf0a5-51da-4b65-bb64-30fec4b3b1d6 None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:03.211748 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45faf0a5-51da-4b65-bb64-30fec4b3b1d6 None admin] RBAC: Authorizing `identity:get_project(project_id=0efa83f60031447ab87be4f6f1af6165)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:03.213957 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45faf0a5-51da-4b65-bb64-30fec4b3b1d6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:03.216312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 91/179] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:03 2022] GET /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:03.217530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38f35c-83e8-4504-9a49-237ae8a16995 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:03.217815 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38f35c-83e8-4504-9a49-237ae8a16995 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:03.218086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38f35c-83e8-4504-9a49-237ae8a16995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:03.219460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 89/180] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:03.220834 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e083fb-773d-4de3-8303-6baf4c609b51 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:03.241236 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ce38f35c-83e8-4504-9a49-237ae8a16995 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:03.242870 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ce38f35c-83e8-4504-9a49-237ae8a16995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:03.260035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:03.270143 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:03.333357 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 92/181] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:03.335519 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabd6e8-f977-40fe-abbe-a9e08b82b3c0 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:03.335952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabd6e8-f977-40fe-abbe-a9e08b82b3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:03.336305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabd6e8-f977-40fe-abbe-a9e08b82b3c0 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:03.338007 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 93/182] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:03.342610 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aabd6e8-f977-40fe-abbe-a9e08b82b3c0 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:03.343139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e083fb-773d-4de3-8303-6baf4c609b51 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:03.387597 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d5f965-093b-41ec-a29f-318bb8e6d49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:03.388501 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5f965-093b-41ec-a29f-318bb8e6d49a None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:03.388783 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5f965-093b-41ec-a29f-318bb8e6d49a None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:03.389175 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5f965-093b-41ec-a29f-318bb8e6d49a None admin] PATH_INFO: `/v3/projects/09ed03f605ac4be88166564d72a5e9e3` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:03.390906 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d5f965-093b-41ec-a29f-318bb8e6d49a None admin] RBAC: Authorizing `identity:get_project(project_id=09ed03f605ac4be88166564d72a5e9e3)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:03.392387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d5f965-093b-41ec-a29f-318bb8e6d49a None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:03.394384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 90/183] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:03 2022] GET /identity/v3/projects/09ed03f605ac4be88166564d72a5e9e3 => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:03.395289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d5f965-093b-41ec-a29f-318bb8e6d49a None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:03.468901 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aabd6e8-f977-40fe-abbe-a9e08b82b3c0 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:03.503709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:03.506657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00becc68-f01d-4a55-bb87-05b3775dafcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:03.507331 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d5f965-093b-41ec-a29f-318bb8e6d49a None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:03.507512 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00becc68-f01d-4a55-bb87-05b3775dafcc None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:03.507772 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00becc68-f01d-4a55-bb87-05b3775dafcc None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:03.508074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00becc68-f01d-4a55-bb87-05b3775dafcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:03.508939 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00becc68-f01d-4a55-bb87-05b3775dafcc None admin] RBAC: Adding query filter params (role.id=f214f8dbc3354cfc81fa2c2f116980e0, scope.project.id=0efa83f60031447ab87be4f6f1af6165, user.id=709a0882f46a4bcaa3f47007f20cfaa1) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:03.509527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00becc68-f01d-4a55-bb87-05b3775dafcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:03.512084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00becc68-f01d-4a55-bb87-05b3775dafcc None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:03.520406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 94/184] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:24:03 2022] GET /identity/v3/role_assignments?user.id=709a0882f46a4bcaa3f47007f20cfaa1&scope.project.id=0efa83f60031447ab87be4f6f1af6165&role.id=f214f8dbc3354cfc81fa2c2f116980e0 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:03.521621 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00117e7-f769-4470-9790-8f5ca350bfad None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:03.522033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00117e7-f769-4470-9790-8f5ca350bfad None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:03.522420 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00117e7-f769-4470-9790-8f5ca350bfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:03.547164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-f00117e7-f769-4470-9790-8f5ca350bfad None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:03.548425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f00117e7-f769-4470-9790-8f5ca350bfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:03.562382 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f5a2c4-8296-443f-80d1-df3bdc12de84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:03.563619 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f5a2c4-8296-443f-80d1-df3bdc12de84 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:03.564122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f5a2c4-8296-443f-80d1-df3bdc12de84 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:03.564835 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f5a2c4-8296-443f-80d1-df3bdc12de84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:03.566230 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:03.585160 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f5a2c4-8296-443f-80d1-df3bdc12de84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:03.596599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-f8f5a2c4-8296-443f-80d1-df3bdc12de84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:24:03.596599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:03.596599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:03.596599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:03.596599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:03.596599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:03.596599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:03.596599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:03.596599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:03.596599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:03.596599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:03.596599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:03.596599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:03.596599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:03.596599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:03.596599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:03.597779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:03.597779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:03.597779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:03.597779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:03.597779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:03.597779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:03.597779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:03.597779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:03.597779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:03.597779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:03.597779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:03.597779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:03.597779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:03.597779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:03.597779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:03.597779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:03.598789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:03.598789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:03.598789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:03.598789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:03.598789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:03.598789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:03.598789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:03.598789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:03.598789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:03.598789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:03.598789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:24:03.598789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:03.601554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 91/185] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:24:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:03.603495 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f5a2c4-8296-443f-80d1-df3bdc12de84 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:03.639807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 95/186] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:03.639807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00117e7-f769-4470-9790-8f5ca350bfad None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:03.747717 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f5a2c4-8296-443f-80d1-df3bdc12de84 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:03.785371 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690059a-d0d1-48ad-a938-e6a1701863c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:03.786169 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690059a-d0d1-48ad-a938-e6a1701863c7 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:03.786509 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690059a-d0d1-48ad-a938-e6a1701863c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:03.787506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690059a-d0d1-48ad-a938-e6a1701863c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:03.791004 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690059a-d0d1-48ad-a938-e6a1701863c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:03.791126 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00117e7-f769-4470-9790-8f5ca350bfad None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:03.791221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690059a-d0d1-48ad-a938-e6a1701863c7 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:03.793969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 92/187] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:03.794597 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690059a-d0d1-48ad-a938-e6a1701863c7 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:03.838475 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:03.845838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0359a-e725-472b-88e1-db84c3a03f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:03.846771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0359a-e725-472b-88e1-db84c3a03f15 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:03.847451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0359a-e725-472b-88e1-db84c3a03f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:03.847992 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0359a-e725-472b-88e1-db84c3a03f15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:03.849081 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e0359a-e725-472b-88e1-db84c3a03f15 None admin] RBAC: Adding query filter params (role.id=6bcb06bf4aa4442ebcfebeadf1af41de, scope.project.id=09ed03f605ac4be88166564d72a5e9e3, user.id=709a0882f46a4bcaa3f47007f20cfaa1) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:03.849637 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e0359a-e725-472b-88e1-db84c3a03f15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:03.851353 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e0359a-e725-472b-88e1-db84c3a03f15 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:03.858925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 96/188] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:24:03 2022] GET /identity/v3/role_assignments?user.id=709a0882f46a4bcaa3f47007f20cfaa1&scope.project.id=09ed03f605ac4be88166564d72a5e9e3&role.id=6bcb06bf4aa4442ebcfebeadf1af41de => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:03.860049 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0359a-e725-472b-88e1-db84c3a03f15 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:03.877893 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:03.928847 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690059a-d0d1-48ad-a938-e6a1701863c7 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:03.975147 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed77106-2461-47ce-a5ae-cd95f3a8239f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:03.976261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed77106-2461-47ce-a5ae-cd95f3a8239f None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:03.976547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed77106-2461-47ce-a5ae-cd95f3a8239f None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:03.976819 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed77106-2461-47ce-a5ae-cd95f3a8239f None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:03.977582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed77106-2461-47ce-a5ae-cd95f3a8239f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:03.977886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed77106-2461-47ce-a5ae-cd95f3a8239f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:03.979308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed77106-2461-47ce-a5ae-cd95f3a8239f None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:03.982944 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0359a-e725-472b-88e1-db84c3a03f15 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:03.988380 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 93/189] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:03.989019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed77106-2461-47ce-a5ae-cd95f3a8239f None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:04.027176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a2fc3-cd16-4520-81bf-c034609b7818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:04.028487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a2fc3-cd16-4520-81bf-c034609b7818 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:04.028978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a2fc3-cd16-4520-81bf-c034609b7818 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:04.029452 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a2fc3-cd16-4520-81bf-c034609b7818 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:04.042619 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a2fc3-cd16-4520-81bf-c034609b7818 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:04.051846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-7c9a2fc3-cd16-4520-81bf-c034609b7818 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:04.051846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:04.051846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:04.051846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:04.051846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:04.051846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:04.051846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:04.051846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:04.051846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:04.051846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:04.051846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:04.051846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:04.051846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:04.051846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:04.051846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:04.051846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:04.053363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:04.053363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:04.053363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:04.053363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:04.053363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:04.053363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:04.053363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:04.053363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:04.053363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:04.053363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:04.053363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:04.053363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:04.053363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:04.053363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:04.053363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:04.053363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:04.054270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:04.054270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:04.054270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:04.054270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:04.054270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:04.054270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:04.054270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:04.054270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:04.054270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:04.054270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:04.054270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:04.054270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:04.056396 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 97/190] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:24:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:04.057901 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48555034-d763-49fa-9155-c2ffc95c4db5 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:04.058194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48555034-d763-49fa-9155-c2ffc95c4db5 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:04.058472 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48555034-d763-49fa-9155-c2ffc95c4db5 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:04.063889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 98/191] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:04.063889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48555034-d763-49fa-9155-c2ffc95c4db5 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:04.095358 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:04.108335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed77106-2461-47ce-a5ae-cd95f3a8239f None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:04.144367 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368b3a9-0160-4546-b6b7-0e98f2f848c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:04.145029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368b3a9-0160-4546-b6b7-0e98f2f848c0 None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:04.145308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368b3a9-0160-4546-b6b7-0e98f2f848c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:04.145579 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368b3a9-0160-4546-b6b7-0e98f2f848c0 None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:04.146330 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368b3a9-0160-4546-b6b7-0e98f2f848c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:04.146914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:04.147766 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368b3a9-0160-4546-b6b7-0e98f2f848c0 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:04.167696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 94/192] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jun 6 10:24:03 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:24:04.168343 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368b3a9-0160-4546-b6b7-0e98f2f848c0 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:04.187408 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48555034-d763-49fa-9155-c2ffc95c4db5 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:04.226543 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da58558-0226-4708-9ed6-00fb98129af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:04.227446 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da58558-0226-4708-9ed6-00fb98129af2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:04.227727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da58558-0226-4708-9ed6-00fb98129af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:04.228026 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da58558-0226-4708-9ed6-00fb98129af2 None admin] PATH_INFO: `/v3/users/2330fa5347a54fb5b897b8096d0013c1` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:04.247345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da58558-0226-4708-9ed6-00fb98129af2 None admin] RBAC: Authorizing `identity:get_user(user_id=2330fa5347a54fb5b897b8096d0013c1)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:04.249677 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da58558-0226-4708-9ed6-00fb98129af2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:04.257040 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 99/193] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:04 2022] GET /identity/v3/users/2330fa5347a54fb5b897b8096d0013c1 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:04.257040 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecb991-d0a4-4952-a507-2cc3fba8e7a1 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:04.257040 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecb991-d0a4-4952-a507-2cc3fba8e7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:04.257040 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecb991-d0a4-4952-a507-2cc3fba8e7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:04.272672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-f4ecb991-d0a4-4952-a507-2cc3fba8e7a1 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:04.274185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f4ecb991-d0a4-4952-a507-2cc3fba8e7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:04.295894 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368b3a9-0160-4546-b6b7-0e98f2f848c0 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:04.341719 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0f93b-e2c9-4575-b83d-66e9ba94a040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:04.343005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0f93b-e2c9-4575-b83d-66e9ba94a040 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:04.343295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0f93b-e2c9-4575-b83d-66e9ba94a040 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:04.343566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0f93b-e2c9-4575-b83d-66e9ba94a040 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:04.344355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e0f93b-e2c9-4575-b83d-66e9ba94a040 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:04.344660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e0f93b-e2c9-4575-b83d-66e9ba94a040 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:04.346062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e0f93b-e2c9-4575-b83d-66e9ba94a040 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:04.346553 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 100/194] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:04.347170 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecb991-d0a4-4952-a507-2cc3fba8e7a1 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:04.359806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 95/195] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:24:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:04.367225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0f93b-e2c9-4575-b83d-66e9ba94a040 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:04.380055 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:04.438671 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:04.496713 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecb991-d0a4-4952-a507-2cc3fba8e7a1 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:04.502746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0f93b-e2c9-4575-b83d-66e9ba94a040 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:04.536189 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdccfa-d766-4017-b9a2-7788cf0c032d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:04.537770 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdccfa-d766-4017-b9a2-7788cf0c032d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:04.537950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdccfa-d766-4017-b9a2-7788cf0c032d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:04.539831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdccfa-d766-4017-b9a2-7788cf0c032d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:04.544209 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f310b73-54bc-402b-aac1-4cae611e7457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:04.545272 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f310b73-54bc-402b-aac1-4cae611e7457 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:04.545686 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f310b73-54bc-402b-aac1-4cae611e7457 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:04.546152 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f310b73-54bc-402b-aac1-4cae611e7457 None admin] PATH_INFO: `/v3/users/2330fa5347a54fb5b897b8096d0013c1` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:04.547506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f310b73-54bc-402b-aac1-4cae611e7457 None admin] RBAC: Authorizing `identity:get_user(user_id=2330fa5347a54fb5b897b8096d0013c1)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:04.549149 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f310b73-54bc-402b-aac1-4cae611e7457 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:04.550958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 96/196] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:04 2022] GET /identity/v3/users/2330fa5347a54fb5b897b8096d0013c1 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:04.556346 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f310b73-54bc-402b-aac1-4cae611e7457 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:04.632739 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdccfa-d766-4017-b9a2-7788cf0c032d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:04.634569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdccfa-d766-4017-b9a2-7788cf0c032d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:04.637720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:04.666510 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f310b73-54bc-402b-aac1-4cae611e7457 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:04.702266 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b405bf8-53e2-4f5d-b61f-69a2e8ffeb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:04.703402 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b405bf8-53e2-4f5d-b61f-69a2e8ffeb72 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:04.703918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b405bf8-53e2-4f5d-b61f-69a2e8ffeb72 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:04.704391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b405bf8-53e2-4f5d-b61f-69a2e8ffeb72 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:04.715077 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:04.716570 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b405bf8-53e2-4f5d-b61f-69a2e8ffeb72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:04.725335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-7b405bf8-53e2-4f5d-b61f-69a2e8ffeb72 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 10:24:04.725335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:04.725335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:04.725335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:04.725335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:04.725335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:04.725335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:04.725335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:04.725335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:04.725335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:04.725335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:04.725335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:04.725335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:04.725335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:04.725335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:04.725335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:04.726431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:04.726431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:04.726431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:04.726431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:04.726431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:04.726431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:04.726431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:04.726431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:04.726431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:04.726431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:04.726431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:04.726431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:04.726431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:04.726431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:04.726431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:04.726431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:04.727213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:04.727213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:04.727213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:04.727213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:04.727213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:04.727213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:04.727213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:04.727213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:04.727213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:04.727213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:04.727213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 10:24:04.727213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:04.729645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 97/197] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:04.730742 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b405bf8-53e2-4f5d-b61f-69a2e8ffeb72 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:04.756120 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 101/198] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:04.756120 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdccfa-d766-4017-b9a2-7788cf0c032d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:04.862052 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdccfa-d766-4017-b9a2-7788cf0c032d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:04.862975 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b405bf8-53e2-4f5d-b61f-69a2e8ffeb72 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:04.897732 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1107cb-34d5-441e-b544-78ffde2fc954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:04.898411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1107cb-34d5-441e-b544-78ffde2fc954 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:04.898686 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1107cb-34d5-441e-b544-78ffde2fc954 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:04.899208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1107cb-34d5-441e-b544-78ffde2fc954 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:04.900504 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1107cb-34d5-441e-b544-78ffde2fc954 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:04.900504 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1107cb-34d5-441e-b544-78ffde2fc954 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:04.902172 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1107cb-34d5-441e-b544-78ffde2fc954 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:04.911114 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff556d02-5918-4a4e-9d50-371c20140c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:04.912713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff556d02-5918-4a4e-9d50-371c20140c7a None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:04.913275 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff556d02-5918-4a4e-9d50-371c20140c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:04.913400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 102/199] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:24:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:04.913950 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff556d02-5918-4a4e-9d50-371c20140c7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:04.914200 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1107cb-34d5-441e-b544-78ffde2fc954 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:04.919729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:04.950139 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:05.006989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff556d02-5918-4a4e-9d50-371c20140c7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:05.009187 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff556d02-5918-4a4e-9d50-371c20140c7a None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:05.042754 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1107cb-34d5-441e-b544-78ffde2fc954 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:05.080826 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdb899-89e9-48b4-8cf8-6965e2669e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:05.081791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdb899-89e9-48b4-8cf8-6965e2669e1c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:05.082076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdb899-89e9-48b4-8cf8-6965e2669e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:05.082347 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdb899-89e9-48b4-8cf8-6965e2669e1c None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:05.083978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defdb899-89e9-48b4-8cf8-6965e2669e1c None admin] RBAC: Authorizing `identity:get_project(project_id=d65d04e96c4b4b87be63be85bb9731a0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:05.086248 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defdb899-89e9-48b4-8cf8-6965e2669e1c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:05.088228 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 103/200] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:04 2022] GET /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:05.088783 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdb899-89e9-48b4-8cf8-6965e2669e1c None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:05.119093 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 98/201] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:05.120381 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff556d02-5918-4a4e-9d50-371c20140c7a None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:05.157526 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:05.191935 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:05.201713 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdb899-89e9-48b4-8cf8-6965e2669e1c None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:05.227102 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff556d02-5918-4a4e-9d50-371c20140c7a None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:05.240325 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8edadf-9d01-48c3-bc04-a008eaf1f608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:05.241395 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8edadf-9d01-48c3-bc04-a008eaf1f608 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:05.241733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8edadf-9d01-48c3-bc04-a008eaf1f608 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:05.242002 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8edadf-9d01-48c3-bc04-a008eaf1f608 None admin] PATH_INFO: `/v3/users/d518e1cdf98a415ebb623b7286bc4086` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:05.243634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8edadf-9d01-48c3-bc04-a008eaf1f608 None admin] RBAC: Authorizing `identity:get_user(user_id=d518e1cdf98a415ebb623b7286bc4086)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:05.244924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8edadf-9d01-48c3-bc04-a008eaf1f608 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:05.246668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 104/202] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:05 2022] GET /identity/v3/users/d518e1cdf98a415ebb623b7286bc4086 => generated 289 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:05.247570 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8edadf-9d01-48c3-bc04-a008eaf1f608 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:05.265483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aff1a0-378b-42f8-861b-6601b12f84f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:05.266425 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aff1a0-378b-42f8-861b-6601b12f84f1 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:05.266922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aff1a0-378b-42f8-861b-6601b12f84f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:05.267386 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aff1a0-378b-42f8-861b-6601b12f84f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:05.268519 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aff1a0-378b-42f8-861b-6601b12f84f1 None admin] RBAC: Adding query filter params (role.id=0f929204eb7b4ea8859338ed2d3ff75b, scope.project.id=d65d04e96c4b4b87be63be85bb9731a0, user.id=2330fa5347a54fb5b897b8096d0013c1) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:05.269025 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aff1a0-378b-42f8-861b-6601b12f84f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:05.270670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aff1a0-378b-42f8-861b-6601b12f84f1 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:05.277945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 99/203] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:24:05 2022] GET /identity/v3/role_assignments?user.id=2330fa5347a54fb5b897b8096d0013c1&scope.project.id=d65d04e96c4b4b87be63be85bb9731a0&role.id=0f929204eb7b4ea8859338ed2d3ff75b => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:05.278995 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aff1a0-378b-42f8-861b-6601b12f84f1 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:05.375665 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8edadf-9d01-48c3-bc04-a008eaf1f608 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:05.387174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:05.391330 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aff1a0-378b-42f8-861b-6601b12f84f1 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:05.415587 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa26802-ed1c-47e1-8c8b-11d6a57e628d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:05.416653 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa26802-ed1c-47e1-8c8b-11d6a57e628d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:05.416936 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa26802-ed1c-47e1-8c8b-11d6a57e628d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:05.417203 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa26802-ed1c-47e1-8c8b-11d6a57e628d None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:05.418461 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa26802-ed1c-47e1-8c8b-11d6a57e628d None admin] RBAC: Authorizing `identity:get_project(project_id=d65d04e96c4b4b87be63be85bb9731a0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:05.420190 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa26802-ed1c-47e1-8c8b-11d6a57e628d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:05.422066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 105/204] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:05 2022] GET /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:05.431153 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa26802-ed1c-47e1-8c8b-11d6a57e628d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:05.445109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc717c-21b3-4344-8a5a-f9ca1707989e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:05.446075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc717c-21b3-4344-8a5a-f9ca1707989e None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:05.446516 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc717c-21b3-4344-8a5a-f9ca1707989e None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:05.446925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc717c-21b3-4344-8a5a-f9ca1707989e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:05.463053 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:05.530148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc717c-21b3-4344-8a5a-f9ca1707989e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:05.531641 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc717c-21b3-4344-8a5a-f9ca1707989e None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:05.546831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa26802-ed1c-47e1-8c8b-11d6a57e628d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:05.588882 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ddbe63-e1bd-41c8-a856-d80c808f72bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:05.589901 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddbe63-e1bd-41c8-a856-d80c808f72bb None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:05.590380 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddbe63-e1bd-41c8-a856-d80c808f72bb None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:05.590899 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddbe63-e1bd-41c8-a856-d80c808f72bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:05.592112 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ddbe63-e1bd-41c8-a856-d80c808f72bb None admin] RBAC: Adding query filter params (role.id=f214f8dbc3354cfc81fa2c2f116980e0, scope.project.id=d65d04e96c4b4b87be63be85bb9731a0, user.id=2330fa5347a54fb5b897b8096d0013c1) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:05.592638 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ddbe63-e1bd-41c8-a856-d80c808f72bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:05.594288 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ddbe63-e1bd-41c8-a856-d80c808f72bb None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:05.601661 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 106/205] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:24:05 2022] GET /identity/v3/role_assignments?user.id=2330fa5347a54fb5b897b8096d0013c1&scope.project.id=d65d04e96c4b4b87be63be85bb9731a0&role.id=f214f8dbc3354cfc81fa2c2f116980e0 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:05.647209 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:05.651551 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 100/206] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:05.657848 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ddbe63-e1bd-41c8-a856-d80c808f72bb None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:05.785537 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:05.796284 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ddbe63-e1bd-41c8-a856-d80c808f72bb None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:05.834971 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704858ee-7c05-4df8-a4c8-175c1496693c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:05.836349 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704858ee-7c05-4df8-a4c8-175c1496693c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:05.836827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704858ee-7c05-4df8-a4c8-175c1496693c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:05.837301 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704858ee-7c05-4df8-a4c8-175c1496693c None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:05.838861 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704858ee-7c05-4df8-a4c8-175c1496693c None admin] RBAC: Authorizing `identity:get_project(project_id=0efa83f60031447ab87be4f6f1af6165)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:05.840742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704858ee-7c05-4df8-a4c8-175c1496693c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:05.843058 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 107/207] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:05 2022] GET /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:05.851811 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc717c-21b3-4344-8a5a-f9ca1707989e None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:05.987399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc717c-21b3-4344-8a5a-f9ca1707989e None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:06.022995 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434d8ef-eff2-4705-8078-e8aa0bbd3369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:06.023715 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434d8ef-eff2-4705-8078-e8aa0bbd3369 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:06.024228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434d8ef-eff2-4705-8078-e8aa0bbd3369 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:06.024508 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434d8ef-eff2-4705-8078-e8aa0bbd3369 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:06.025403 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434d8ef-eff2-4705-8078-e8aa0bbd3369 None admin] RBAC: Adding query filter params (role.id=803bb32940ee4c78ab77b3d63cd7ac0f, scope.project.id=0efa83f60031447ab87be4f6f1af6165, user.id=d518e1cdf98a415ebb623b7286bc4086) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:06.025720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434d8ef-eff2-4705-8078-e8aa0bbd3369 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:06.027215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434d8ef-eff2-4705-8078-e8aa0bbd3369 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:06.033266 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 101/208] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:24:05 2022] GET /identity/v3/role_assignments?user.id=d518e1cdf98a415ebb623b7286bc4086&scope.project.id=0efa83f60031447ab87be4f6f1af6165&role.id=803bb32940ee4c78ab77b3d63cd7ac0f => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:06.879175 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b65fb0-46da-48d1-8c5d-98ba4125d893 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:06.879689 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b65fb0-46da-48d1-8c5d-98ba4125d893 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:06.879763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b65fb0-46da-48d1-8c5d-98ba4125d893 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:06.881924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 108/209] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:06.887544 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce0699-a627-44b3-a952-10a2d4f85de0 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:06.887867 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce0699-a627-44b3-a952-10a2d4f85de0 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:06.888152 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce0699-a627-44b3-a952-10a2d4f85de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:06.894218 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b7d25-bd94-4d56-9baf-9a4efc1cf412 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:06.894765 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b7d25-bd94-4d56-9baf-9a4efc1cf412 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:06.895242 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b7d25-bd94-4d56-9baf-9a4efc1cf412 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:06.896988 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 109/210] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:24:06.902374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04f8fc-547f-4dac-8a61-9987dad937da None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:06.902882 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04f8fc-547f-4dac-8a61-9987dad937da None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:06.903369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04f8fc-547f-4dac-8a61-9987dad937da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:06.907186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-5fce0699-a627-44b3-a952-10a2d4f85de0 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:06.908423 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-5fce0699-a627-44b3-a952-10a2d4f85de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:06.920727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2a04f8fc-547f-4dac-8a61-9987dad937da None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:06.922175 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2a04f8fc-547f-4dac-8a61-9987dad937da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:06.973588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:06.983583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 102/211] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:07.002129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 110/212] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 10:24:07.009603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce0699-a627-44b3-a952-10a2d4f85de0 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:07.026428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04f8fc-547f-4dac-8a61-9987dad937da None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:07.113230 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce0699-a627-44b3-a952-10a2d4f85de0 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:07.123636 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:07.134137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04f8fc-547f-4dac-8a61-9987dad937da None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:07.152648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76efbf6-84cc-4117-8ec5-445387b5ce26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:07.153444 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76efbf6-84cc-4117-8ec5-445387b5ce26 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:07.153726 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76efbf6-84cc-4117-8ec5-445387b5ce26 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:07.154001 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76efbf6-84cc-4117-8ec5-445387b5ce26 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:07.155515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76efbf6-84cc-4117-8ec5-445387b5ce26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:07.157446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76efbf6-84cc-4117-8ec5-445387b5ce26 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:07.163317 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 103/213] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:07.166625 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76efbf6-84cc-4117-8ec5-445387b5ce26 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:07.170642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff08a9-a2e8-41ce-bab8-5088bc51ebf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:07.171451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff08a9-a2e8-41ce-bab8-5088bc51ebf0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:07.171890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff08a9-a2e8-41ce-bab8-5088bc51ebf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:07.172168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff08a9-a2e8-41ce-bab8-5088bc51ebf0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:07.183535 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff08a9-a2e8-41ce-bab8-5088bc51ebf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:07.191413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-63ff08a9-a2e8-41ce-bab8-5088bc51ebf0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:07.191413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:07.191413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:07.191413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:07.191413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:07.191413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:07.191413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:07.191413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:07.191413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:07.191413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:07.191413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:07.191413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:07.191413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:07.191413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:07.191413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:07.191413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:07.192305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:07.192305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:07.192305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:07.192305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:07.192305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:07.192305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:07.192305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:07.192305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:07.192305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:07.192305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:07.192305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:07.192305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:07.192305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:07.192305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:07.192305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:07.192305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:07.193617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:07.193617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:07.193617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:07.193617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:07.193617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:07.193617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:07.193617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:07.193617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:07.193617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:07.193617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:07.193617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:07.193617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:07.193617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 111/214] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:07.199937 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff08a9-a2e8-41ce-bab8-5088bc51ebf0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:07.241925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:07.305787 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76efbf6-84cc-4117-8ec5-445387b5ce26 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:07.325643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff08a9-a2e8-41ce-bab8-5088bc51ebf0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:07.353416 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa22d7-3f00-4e1d-81a8-1f9d0cffc1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:07.354428 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa22d7-3f00-4e1d-81a8-1f9d0cffc1ce None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:07.354983 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa22d7-3f00-4e1d-81a8-1f9d0cffc1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:07.355448 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa22d7-3f00-4e1d-81a8-1f9d0cffc1ce None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:07.356447 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa22d7-3f00-4e1d-81a8-1f9d0cffc1ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:07.358123 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa22d7-3f00-4e1d-81a8-1f9d0cffc1ce None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:07.362766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c474b-02a0-47ad-bf30-158aceaf9fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:07.363441 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c474b-02a0-47ad-bf30-158aceaf9fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:07.363721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c474b-02a0-47ad-bf30-158aceaf9fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:07.364038 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c474b-02a0-47ad-bf30-158aceaf9fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:07.364781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c474b-02a0-47ad-bf30-158aceaf9fa7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:07.365612 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c474b-02a0-47ad-bf30-158aceaf9fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:07.367474 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c474b-02a0-47ad-bf30-158aceaf9fa7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:07.376414 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 112/215] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:24:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:07.384566 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c474b-02a0-47ad-bf30-158aceaf9fa7 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:07.397098 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 104/216] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jun 6 10:24:07 2022] POST /identity/v3/users => generated 297 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:24:07.414796 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:07.516538 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c474b-02a0-47ad-bf30-158aceaf9fa7 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:07.558853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6c88e-a4d2-4806-bc06-3254a0bf4e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:07.559749 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6c88e-a4d2-4806-bc06-3254a0bf4e66 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:07.560056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6c88e-a4d2-4806-bc06-3254a0bf4e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:07.560331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6c88e-a4d2-4806-bc06-3254a0bf4e66 None admin] PATH_INFO: `/v3/users/222963455d3e46e5985ab76dc39a4471` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:07.577235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6c88e-a4d2-4806-bc06-3254a0bf4e66 None admin] RBAC: Authorizing `identity:get_user(user_id=222963455d3e46e5985ab76dc39a4471)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:07.578727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6c88e-a4d2-4806-bc06-3254a0bf4e66 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:07.580947 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 113/217] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:07 2022] GET /identity/v3/users/222963455d3e46e5985ab76dc39a4471 => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:07.587320 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa22d7-3f00-4e1d-81a8-1f9d0cffc1ce None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:07.704177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa22d7-3f00-4e1d-81a8-1f9d0cffc1ce None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:07.722982 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:07.745573 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522308fa-924f-4335-a015-6ac7cbe82a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:07.746228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522308fa-924f-4335-a015-6ac7cbe82a9c None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:07.746509 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522308fa-924f-4335-a015-6ac7cbe82a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:07.746785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522308fa-924f-4335-a015-6ac7cbe82a9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:07.822782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522308fa-924f-4335-a015-6ac7cbe82a9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:07.824390 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522308fa-924f-4335-a015-6ac7cbe82a9c None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:07.934313 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 105/218] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:07.939774 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6c88e-a4d2-4806-bc06-3254a0bf4e66 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:07.949447 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eec78-1055-42b0-b44e-a15f15d9f270 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:07.949747 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eec78-1055-42b0-b44e-a15f15d9f270 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:07.950016 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eec78-1055-42b0-b44e-a15f15d9f270 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:07.951837 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 106/219] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:07.957519 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cadb5-1a0b-4ade-abcf-f822f44cfa2c None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:07.957976 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cadb5-1a0b-4ade-abcf-f822f44cfa2c None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:07.958396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cadb5-1a0b-4ade-abcf-f822f44cfa2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:07.978464 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-482cadb5-1a0b-4ade-abcf-f822f44cfa2c None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:07.979845 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-482cadb5-1a0b-4ade-abcf-f822f44cfa2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:08.000891 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:08.032544 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:08.061213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 107/220] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 10:24:08.086682 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6c88e-a4d2-4806-bc06-3254a0bf4e66 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:08.102065 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cadb5-1a0b-4ade-abcf-f822f44cfa2c None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:08.132521 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f796b4f-809e-494b-8d71-20599b0247b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:08.133509 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f796b4f-809e-494b-8d71-20599b0247b6 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:08.133792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f796b4f-809e-494b-8d71-20599b0247b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:08.134067 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f796b4f-809e-494b-8d71-20599b0247b6 None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:08.135268 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f796b4f-809e-494b-8d71-20599b0247b6 None admin] RBAC: Authorizing `identity:get_project(project_id=d65d04e96c4b4b87be63be85bb9731a0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:08.136881 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f796b4f-809e-494b-8d71-20599b0247b6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:08.139017 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 114/221] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:07 2022] GET /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:08.148805 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f796b4f-809e-494b-8d71-20599b0247b6 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:08.215406 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cadb5-1a0b-4ade-abcf-f822f44cfa2c None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:08.253486 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8fb4a-8137-427f-bee2-c9c5c97ec484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:08.254753 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8fb4a-8137-427f-bee2-c9c5c97ec484 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:08.255236 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8fb4a-8137-427f-bee2-c9c5c97ec484 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:08.255809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8fb4a-8137-427f-bee2-c9c5c97ec484 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:08.267171 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8fb4a-8137-427f-bee2-c9c5c97ec484 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:08.282763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-79f8fb4a-8137-427f-bee2-c9c5c97ec484 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:24:08.282763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:08.282763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:08.282763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:08.282763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:08.282763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:08.282763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:08.282763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:08.282763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:08.282763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:08.282763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:08.282763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:08.282763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:08.282763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:08.282763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:08.282763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:08.283699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:08.283699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:08.283699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:08.283699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:08.283699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:08.283699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:08.283699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:08.283699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:08.283699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:08.283699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:08.283699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:08.283699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:08.283699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:08.283699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:08.283699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:08.283699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:08.284541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:08.284541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:08.284541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:08.284541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:08.284541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:08.284541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:08.284541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:08.284541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:08.284541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:08.284541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:08.284541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:24:08.284541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:08.287945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 108/222] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:24:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:24:08.293920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8fb4a-8137-427f-bee2-c9c5c97ec484 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:08.316176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:08.379942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f796b4f-809e-494b-8d71-20599b0247b6 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:08.424739 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d062b-96f8-4e18-8efe-f6fdf597c365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:08.425395 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d062b-96f8-4e18-8efe-f6fdf597c365 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:08.425674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d062b-96f8-4e18-8efe-f6fdf597c365 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:08.425950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d062b-96f8-4e18-8efe-f6fdf597c365 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:08.426878 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d062b-96f8-4e18-8efe-f6fdf597c365 None admin] RBAC: Adding query filter params (role.id=6bcb06bf4aa4442ebcfebeadf1af41de, scope.project.id=d65d04e96c4b4b87be63be85bb9731a0, user.id=222963455d3e46e5985ab76dc39a4471) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:08.427213 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d062b-96f8-4e18-8efe-f6fdf597c365 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:08.429339 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d062b-96f8-4e18-8efe-f6fdf597c365 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:08.433196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8fb4a-8137-427f-bee2-c9c5c97ec484 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:08.435721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 115/223] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:24:08 2022] GET /identity/v3/role_assignments?user.id=222963455d3e46e5985ab76dc39a4471&scope.project.id=d65d04e96c4b4b87be63be85bb9731a0&role.id=6bcb06bf4aa4442ebcfebeadf1af41de => generated 259 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:08.445375 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacd039-d7d4-455f-89a2-960915fb72cc None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:08.445736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacd039-d7d4-455f-89a2-960915fb72cc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:08.446086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacd039-d7d4-455f-89a2-960915fb72cc None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:08.447774 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 116/224] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:08 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:08.453620 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c314a8e-0774-4090-9180-7e61cf60677f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:08.453972 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c314a8e-0774-4090-9180-7e61cf60677f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:08.454413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c314a8e-0774-4090-9180-7e61cf60677f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:08.462376 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:08.476248 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12487fa8-7f06-4367-ae12-b85f8495f0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:08.477284 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12487fa8-7f06-4367-ae12-b85f8495f0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:08.477770 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12487fa8-7f06-4367-ae12-b85f8495f0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:08.478263 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12487fa8-7f06-4367-ae12-b85f8495f0c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:08.479225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12487fa8-7f06-4367-ae12-b85f8495f0c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:08.480352 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12487fa8-7f06-4367-ae12-b85f8495f0c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:08.482160 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12487fa8-7f06-4367-ae12-b85f8495f0c9 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:08.488500 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-1c314a8e-0774-4090-9180-7e61cf60677f None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:08.489518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1c314a8e-0774-4090-9180-7e61cf60677f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:08.492913 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 109/225] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:08.498969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12487fa8-7f06-4367-ae12-b85f8495f0c9 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:08.666436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 117/226] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:08.679928 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:08.696090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c314a8e-0774-4090-9180-7e61cf60677f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:08.712929 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12487fa8-7f06-4367-ae12-b85f8495f0c9 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:08.775890 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5649f2-0a7f-4dd3-ae13-800210971b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:08.776695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5649f2-0a7f-4dd3-ae13-800210971b22 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:08.776971 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5649f2-0a7f-4dd3-ae13-800210971b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:08.777242 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5649f2-0a7f-4dd3-ae13-800210971b22 None admin] PATH_INFO: `/v3/users/2330fa5347a54fb5b897b8096d0013c1` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:08.778727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5649f2-0a7f-4dd3-ae13-800210971b22 None admin] RBAC: Authorizing `identity:get_user(user_id=2330fa5347a54fb5b897b8096d0013c1)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:08.780207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5649f2-0a7f-4dd3-ae13-800210971b22 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:08.782186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 110/227] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:08 2022] GET /identity/v3/users/2330fa5347a54fb5b897b8096d0013c1 => generated 283 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:08.787739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5649f2-0a7f-4dd3-ae13-800210971b22 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:08.844119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c314a8e-0774-4090-9180-7e61cf60677f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:08.875546 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:08.897900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc576d68-0931-4e5e-934c-538e9bba659f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:08.898730 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc576d68-0931-4e5e-934c-538e9bba659f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:08.899276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc576d68-0931-4e5e-934c-538e9bba659f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:08.899546 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc576d68-0931-4e5e-934c-538e9bba659f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:08.918110 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc576d68-0931-4e5e-934c-538e9bba659f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:08.936530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-cc576d68-0931-4e5e-934c-538e9bba659f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:08.936530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:08.936530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:08.936530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:08.936530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:08.936530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:08.936530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:08.936530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:08.936530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:08.936530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:08.936530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:08.936530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:08.936530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:08.936530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:08.936530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:08.936530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:08.937479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:08.937479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:08.937479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:08.937479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:08.937479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:08.937479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:08.937479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:08.937479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:08.937479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:08.937479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:08.937479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:08.937479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:08.937479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:08.937479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:08.937479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:08.937479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:08.938255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:08.938255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:08.938255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:08.938255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:08.938255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:08.938255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:08.938255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:08.938255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:08.938255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:08.938255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:08.938255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:08.938255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:08.944109 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 118/228] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:24:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:08.964863 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc576d68-0931-4e5e-934c-538e9bba659f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:08.967815 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5649f2-0a7f-4dd3-ae13-800210971b22 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:09.012823 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd00220-b4b1-4242-b9d7-e6118da3583e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:09.013724 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd00220-b4b1-4242-b9d7-e6118da3583e None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:09.014049 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd00220-b4b1-4242-b9d7-e6118da3583e None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:09.014323 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd00220-b4b1-4242-b9d7-e6118da3583e None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:09.015495 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd00220-b4b1-4242-b9d7-e6118da3583e None admin] RBAC: Authorizing `identity:get_project(project_id=d65d04e96c4b4b87be63be85bb9731a0)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:09.017227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd00220-b4b1-4242-b9d7-e6118da3583e None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:09.019207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 111/229] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:08 2022] GET /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:09.026420 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd00220-b4b1-4242-b9d7-e6118da3583e None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:09.036540 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:09.147892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd00220-b4b1-4242-b9d7-e6118da3583e None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:09.187121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f581425-b6a9-4a24-afdb-629841594a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:09.188041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f581425-b6a9-4a24-afdb-629841594a97 None admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:09.188320 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f581425-b6a9-4a24-afdb-629841594a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:09.188592 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f581425-b6a9-4a24-afdb-629841594a97 None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0/users/2330fa5347a54fb5b897b8096d0013c1/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:09.195833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f581425-b6a9-4a24-afdb-629841594a97 None admin] RBAC: Authorizing `identity:create_grant(project_id=d65d04e96c4b4b87be63be85bb9731a0, user_id=2330fa5347a54fb5b897b8096d0013c1, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:09.195833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f581425-b6a9-4a24-afdb-629841594a97 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:09.216157 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 112/230] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:24:09 2022] PUT /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0/users/2330fa5347a54fb5b897b8096d0013c1/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:24:09.254158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc576d68-0931-4e5e-934c-538e9bba659f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:09.316372 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81a023-0ceb-4ade-abd3-55135eeb7665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:09.317517 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81a023-0ceb-4ade-abd3-55135eeb7665 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:09.318040 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81a023-0ceb-4ade-abd3-55135eeb7665 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:09.318515 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81a023-0ceb-4ade-abd3-55135eeb7665 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:09.326035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81a023-0ceb-4ade-abd3-55135eeb7665 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:09.326701 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81a023-0ceb-4ade-abd3-55135eeb7665 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:09.328973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81a023-0ceb-4ade-abd3-55135eeb7665 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:09.338559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 119/231] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:24:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:09.344679 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f581425-b6a9-4a24-afdb-629841594a97 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:09.436431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:09.478073 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f581425-b6a9-4a24-afdb-629841594a97 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:09.514207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4576f92-11c3-4383-88e1-cf63eca08b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:09.515081 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4576f92-11c3-4383-88e1-cf63eca08b80 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:09.515365 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4576f92-11c3-4383-88e1-cf63eca08b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:09.515688 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4576f92-11c3-4383-88e1-cf63eca08b80 None admin] PATH_INFO: `/v3/users/2330fa5347a54fb5b897b8096d0013c1` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:09.516966 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4576f92-11c3-4383-88e1-cf63eca08b80 None admin] RBAC: Authorizing `identity:get_user(user_id=2330fa5347a54fb5b897b8096d0013c1)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:09.518406 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4576f92-11c3-4383-88e1-cf63eca08b80 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:09.521969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 113/232] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:09 2022] GET /identity/v3/users/2330fa5347a54fb5b897b8096d0013c1 => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:09.527044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81a023-0ceb-4ade-abd3-55135eeb7665 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:09.594502 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:09.645125 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81a023-0ceb-4ade-abd3-55135eeb7665 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:09.686513 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d4f09-9c20-4602-9a86-50d66edbe8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:09.687745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d4f09-9c20-4602-9a86-50d66edbe8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:09.688263 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d4f09-9c20-4602-9a86-50d66edbe8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:09.688732 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d4f09-9c20-4602-9a86-50d66edbe8e2 None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:09.692521 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d4f09-9c20-4602-9a86-50d66edbe8e2 None admin] RBAC: Authorizing `identity:get_project(project_id=d65d04e96c4b4b87be63be85bb9731a0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:09.694982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d4f09-9c20-4602-9a86-50d66edbe8e2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:09.696938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 120/233] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:09 2022] GET /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:09.702712 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4576f92-11c3-4383-88e1-cf63eca08b80 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:09.836991 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4576f92-11c3-4383-88e1-cf63eca08b80 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:09.880227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b568c89-3c4a-4c89-a432-6869b34353d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:09.882141 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b568c89-3c4a-4c89-a432-6869b34353d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:09.882623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b568c89-3c4a-4c89-a432-6869b34353d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:09.883137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b568c89-3c4a-4c89-a432-6869b34353d0 None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0/users/2330fa5347a54fb5b897b8096d0013c1/roles/f214f8dbc3354cfc81fa2c2f116980e0` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:09.885686 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b568c89-3c4a-4c89-a432-6869b34353d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d65d04e96c4b4b87be63be85bb9731a0, user_id=2330fa5347a54fb5b897b8096d0013c1, role_id=f214f8dbc3354cfc81fa2c2f116980e0)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:09.887377 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b568c89-3c4a-4c89-a432-6869b34353d0 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:09.894477 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:09.919846 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 114/234] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:24:09 2022] PUT /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0/users/2330fa5347a54fb5b897b8096d0013c1/roles/f214f8dbc3354cfc81fa2c2f116980e0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:24:10.344756 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683cc39-09f1-44fe-b151-291d3bc23780 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:10.345655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683cc39-09f1-44fe-b151-291d3bc23780 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:10.345936 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683cc39-09f1-44fe-b151-291d3bc23780 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:10.347470 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 121/235] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:10.353208 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c48ac9-e59e-4d48-be11-d79daa1847bb None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:10.353991 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c48ac9-e59e-4d48-be11-d79daa1847bb None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:10.354276 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c48ac9-e59e-4d48-be11-d79daa1847bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:10.388022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-49c48ac9-e59e-4d48-be11-d79daa1847bb None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:10.391546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-49c48ac9-e59e-4d48-be11-d79daa1847bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:10.486109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 115/236] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:10.524434 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683cc39-09f1-44fe-b151-291d3bc23780 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:10.663073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683cc39-09f1-44fe-b151-291d3bc23780 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:10.695076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:10.707931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cc6c3-2034-4651-9984-b351a44efde4 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:10.707931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cc6c3-2034-4651-9984-b351a44efde4 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:10.707931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cc6c3-2034-4651-9984-b351a44efde4 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:10.708569 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 116/237] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:24:10.709953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8628a4d-744e-4e12-8f14-c105ada3c808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:10.710767 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8628a4d-744e-4e12-8f14-c105ada3c808 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:10.711414 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8628a4d-744e-4e12-8f14-c105ada3c808 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:10.712042 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8628a4d-744e-4e12-8f14-c105ada3c808 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:10.715711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787dd284-e238-421a-97c0-d5ed25ec4e29 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:10.716185 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787dd284-e238-421a-97c0-d5ed25ec4e29 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:10.716604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787dd284-e238-421a-97c0-d5ed25ec4e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:10.730503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8628a4d-744e-4e12-8f14-c105ada3c808 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:10.738499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-b8628a4d-744e-4e12-8f14-c105ada3c808 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 10:24:10.738499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:10.738499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:10.738499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:10.738499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:10.738499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:10.738499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:10.738499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:10.738499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:10.738499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:10.738499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:10.738499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:10.738499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:10.738499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:10.738499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:10.738499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:10.739370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:10.739370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:10.739370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:10.739370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:10.739370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:10.739370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:10.739370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:10.739370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:10.739370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:10.739370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:10.739370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:10.739370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:10.739370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:10.739370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:10.739370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:10.739370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:10.741265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:10.741265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:10.741265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:10.741265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:10.741265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:10.741265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:10.741265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:10.741265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:10.741265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:10.741265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:10.741265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 10:24:10.741265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:10.741265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 122/238] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:10.746177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-787dd284-e238-421a-97c0-d5ed25ec4e29 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:10.747424 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-787dd284-e238-421a-97c0-d5ed25ec4e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:10.748923 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8628a4d-744e-4e12-8f14-c105ada3c808 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:10.865455 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 117/239] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 06 10:24:10.889237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787dd284-e238-421a-97c0-d5ed25ec4e29 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:10.933682 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8628a4d-744e-4e12-8f14-c105ada3c808 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:10.982051 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:10.998102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad5dae-4112-4785-b69c-539cfe635ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:10.998779 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad5dae-4112-4785-b69c-539cfe635ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:10.999057 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad5dae-4112-4785-b69c-539cfe635ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:10.999329 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad5dae-4112-4785-b69c-539cfe635ce5 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:11.002178 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad5dae-4112-4785-b69c-539cfe635ce5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:11.002178 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad5dae-4112-4785-b69c-539cfe635ce5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:11.002178 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad5dae-4112-4785-b69c-539cfe635ce5 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:11.014653 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 123/240] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:24:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:11.020498 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad5dae-4112-4785-b69c-539cfe635ce5 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:11.073920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787dd284-e238-421a-97c0-d5ed25ec4e29 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:11.098082 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:11.139833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1564a-3c77-432f-9879-c0498abac757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:11.139833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1564a-3c77-432f-9879-c0498abac757 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:11.139833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1564a-3c77-432f-9879-c0498abac757 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:11.139833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1564a-3c77-432f-9879-c0498abac757 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:11.140282 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1564a-3c77-432f-9879-c0498abac757 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:11.141579 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1564a-3c77-432f-9879-c0498abac757 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:11.143300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 118/241] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:11.155570 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1564a-3c77-432f-9879-c0498abac757 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:11.188989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad5dae-4112-4785-b69c-539cfe635ce5 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:11.225060 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5abe2-08dd-44c9-9084-4ceda036765e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:11.226252 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5abe2-08dd-44c9-9084-4ceda036765e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:11.226716 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5abe2-08dd-44c9-9084-4ceda036765e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:11.227196 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5abe2-08dd-44c9-9084-4ceda036765e None admin] PATH_INFO: `/v3/users/ccb58e68020044078c1a40b3fe3560b0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:11.241922 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5abe2-08dd-44c9-9084-4ceda036765e None admin] RBAC: Authorizing `identity:get_user(user_id=ccb58e68020044078c1a40b3fe3560b0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:11.243666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5abe2-08dd-44c9-9084-4ceda036765e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:11.245612 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 124/242] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:11 2022] GET /identity/v3/users/ccb58e68020044078c1a40b3fe3560b0 => generated 297 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:11.247204 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861afea0-a94b-4692-a0c9-d1a3a7c3b728 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:11.247702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861afea0-a94b-4692-a0c9-d1a3a7c3b728 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:11.248183 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861afea0-a94b-4692-a0c9-d1a3a7c3b728 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:11.249597 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 125/243] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:11.251268 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861afea0-a94b-4692-a0c9-d1a3a7c3b728 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:11.268002 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1564a-3c77-432f-9879-c0498abac757 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:11.294750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:11.308868 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226a805-d993-4210-86f9-3996e8e58b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:11.309579 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226a805-d993-4210-86f9-3996e8e58b98 None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:11.309868 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226a805-d993-4210-86f9-3996e8e58b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:11.310142 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226a805-d993-4210-86f9-3996e8e58b98 None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:11.310863 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226a805-d993-4210-86f9-3996e8e58b98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:11.312492 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226a805-d993-4210-86f9-3996e8e58b98 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:11.341647 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 119/244] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jun 6 10:24:11 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:24:11.343251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e056c7d-921e-447a-800f-0451ee03b826 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:11.343547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e056c7d-921e-447a-800f-0451ee03b826 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:11.345950 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e056c7d-921e-447a-800f-0451ee03b826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:11.367454 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861afea0-a94b-4692-a0c9-d1a3a7c3b728 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:11.388488 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:11.393941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-9e056c7d-921e-447a-800f-0451ee03b826 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:11.400882 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-9e056c7d-921e-447a-800f-0451ee03b826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:11.465724 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77debc46-b3a5-42a2-aa78-4eb97a6d52db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:11.466363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77debc46-b3a5-42a2-aa78-4eb97a6d52db None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:11.466648 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77debc46-b3a5-42a2-aa78-4eb97a6d52db None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:11.466922 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77debc46-b3a5-42a2-aa78-4eb97a6d52db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:11.554177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 120/245] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:11.589850 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e056c7d-921e-447a-800f-0451ee03b826 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:11.593595 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77debc46-b3a5-42a2-aa78-4eb97a6d52db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:11.595648 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77debc46-b3a5-42a2-aa78-4eb97a6d52db None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:11.705861 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 126/246] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:11.712385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77debc46-b3a5-42a2-aa78-4eb97a6d52db None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:11.714522 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:11.724472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e056c7d-921e-447a-800f-0451ee03b826 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:11.740738 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:11.771095 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9be4b8-0b6b-4c6e-a571-1b50479a8239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:11.772844 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9be4b8-0b6b-4c6e-a571-1b50479a8239 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:11.773327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9be4b8-0b6b-4c6e-a571-1b50479a8239 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:11.775354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9be4b8-0b6b-4c6e-a571-1b50479a8239 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:11.787828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9be4b8-0b6b-4c6e-a571-1b50479a8239 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:11.796227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-3c9be4b8-0b6b-4c6e-a571-1b50479a8239 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:11.796227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:11.796227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:11.796227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:11.796227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:11.796227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:11.796227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:11.796227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:11.796227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:11.796227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:11.796227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:11.796227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:11.796227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:11.796227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:11.796227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:11.796227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:11.797198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:11.797198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:11.797198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:11.797198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:11.797198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:11.797198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:11.797198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:11.797198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:11.797198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:11.797198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:11.797198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:11.797198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:11.797198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:11.797198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:11.797198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:11.797198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:11.797979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:11.797979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:11.797979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:11.797979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:11.797979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:11.797979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:11.797979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:11.797979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:11.797979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:11.797979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:11.797979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:11.797979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:11.800787 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 121/247] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:11.806531 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9be4b8-0b6b-4c6e-a571-1b50479a8239 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:11.821241 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77debc46-b3a5-42a2-aa78-4eb97a6d52db None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:11.856385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76794cb0-52b3-4d53-b47b-13e4d5c1556b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:11.857257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76794cb0-52b3-4d53-b47b-13e4d5c1556b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:11.857559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76794cb0-52b3-4d53-b47b-13e4d5c1556b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:11.857859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76794cb0-52b3-4d53-b47b-13e4d5c1556b None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:11.859926 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76794cb0-52b3-4d53-b47b-13e4d5c1556b None admin] RBAC: Authorizing `identity:get_project(project_id=d65d04e96c4b4b87be63be85bb9731a0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:11.861378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76794cb0-52b3-4d53-b47b-13e4d5c1556b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:11.863089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 127/248] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:11 2022] GET /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:11.869489 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76794cb0-52b3-4d53-b47b-13e4d5c1556b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:11.921963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9be4b8-0b6b-4c6e-a571-1b50479a8239 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:11.945877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:11.963454 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025e3b1-61c2-4f41-9a04-eb25921d18e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:11.964477 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025e3b1-61c2-4f41-9a04-eb25921d18e2 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:11.964983 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025e3b1-61c2-4f41-9a04-eb25921d18e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:11.965518 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025e3b1-61c2-4f41-9a04-eb25921d18e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:11.966591 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025e3b1-61c2-4f41-9a04-eb25921d18e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:11.967092 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025e3b1-61c2-4f41-9a04-eb25921d18e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:11.968809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025e3b1-61c2-4f41-9a04-eb25921d18e2 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:11.979963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 122/249] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:24:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:11.980118 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76794cb0-52b3-4d53-b47b-13e4d5c1556b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:11.986133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025e3b1-61c2-4f41-9a04-eb25921d18e2 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:12.016446 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f136b-f16e-4e49-a67f-d822ffcb32dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:12.016981 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f136b-f16e-4e49-a67f-d822ffcb32dd None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:12.017262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f136b-f16e-4e49-a67f-d822ffcb32dd None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:12.017534 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f136b-f16e-4e49-a67f-d822ffcb32dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:12.018634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f136b-f16e-4e49-a67f-d822ffcb32dd None admin] RBAC: Adding query filter params (role.id=803bb32940ee4c78ab77b3d63cd7ac0f, scope.project.id=d65d04e96c4b4b87be63be85bb9731a0, user.id=ccb58e68020044078c1a40b3fe3560b0) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:12.018634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f136b-f16e-4e49-a67f-d822ffcb32dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:12.020325 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f136b-f16e-4e49-a67f-d822ffcb32dd None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:12.028760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:12.031055 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 128/250] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:24:11 2022] GET /identity/v3/role_assignments?user.id=ccb58e68020044078c1a40b3fe3560b0&scope.project.id=d65d04e96c4b4b87be63be85bb9731a0&role.id=803bb32940ee4c78ab77b3d63cd7ac0f => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:12.032580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ada6ff-e7f7-40a3-bd6d-f454148162c6 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:12.032874 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ada6ff-e7f7-40a3-bd6d-f454148162c6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:12.033211 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ada6ff-e7f7-40a3-bd6d-f454148162c6 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:12.040477 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 129/251] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:12.042729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e87d78-be7c-4374-84f9-6a97bd45824d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:12.043505 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e87d78-be7c-4374-84f9-6a97bd45824d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:12.043801 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e87d78-be7c-4374-84f9-6a97bd45824d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:12.069328 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-17e87d78-be7c-4374-84f9-6a97bd45824d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:12.070846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-17e87d78-be7c-4374-84f9-6a97bd45824d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:12.125538 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025e3b1-61c2-4f41-9a04-eb25921d18e2 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:12.155601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 130/252] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 10:24:12.171874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56885e79-793d-4c64-9229-350902edb72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:12.172764 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56885e79-793d-4c64-9229-350902edb72f None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:12.173042 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56885e79-793d-4c64-9229-350902edb72f None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:12.173318 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56885e79-793d-4c64-9229-350902edb72f None admin] PATH_INFO: `/v3/users/222963455d3e46e5985ab76dc39a4471` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:12.174716 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56885e79-793d-4c64-9229-350902edb72f None admin] RBAC: Authorizing `identity:get_user(user_id=222963455d3e46e5985ab76dc39a4471)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:12.176368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56885e79-793d-4c64-9229-350902edb72f None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:12.178148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 123/253] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:11 2022] GET /identity/v3/users/222963455d3e46e5985ab76dc39a4471 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:12.183816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e87d78-be7c-4374-84f9-6a97bd45824d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:12.193104 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56885e79-793d-4c64-9229-350902edb72f None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:12.256120 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:12.301160 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e87d78-be7c-4374-84f9-6a97bd45824d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:12.308352 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:12.312167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56885e79-793d-4c64-9229-350902edb72f None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:12.342758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa8cbf-a3ac-4fbc-b4e7-d4b0466102a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:12.344034 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa8cbf-a3ac-4fbc-b4e7-d4b0466102a0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:12.344547 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa8cbf-a3ac-4fbc-b4e7-d4b0466102a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:12.345020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa8cbf-a3ac-4fbc-b4e7-d4b0466102a0 None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:12.346483 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa8cbf-a3ac-4fbc-b4e7-d4b0466102a0 None admin] RBAC: Authorizing `identity:get_project(project_id=d65d04e96c4b4b87be63be85bb9731a0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:12.348352 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa8cbf-a3ac-4fbc-b4e7-d4b0466102a0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:12.351207 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 131/254] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:12 2022] GET /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:12.357349 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa8cbf-a3ac-4fbc-b4e7-d4b0466102a0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:12.421072 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e91b13-c865-4198-88fc-f3ff655d8238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:12.421874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e91b13-c865-4198-88fc-f3ff655d8238 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:12.422157 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e91b13-c865-4198-88fc-f3ff655d8238 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:12.422431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e91b13-c865-4198-88fc-f3ff655d8238 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:12.448785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e91b13-c865-4198-88fc-f3ff655d8238 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:12.457225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-78e91b13-c865-4198-88fc-f3ff655d8238 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:24:12.457225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:12.457225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:12.457225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:12.457225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:12.457225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:12.457225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:12.457225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:12.457225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:12.457225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:12.457225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:12.457225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:12.457225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:12.457225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:12.457225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:12.457225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:12.458108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:12.458108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:12.458108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:12.458108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:12.458108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:12.458108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:12.458108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:12.458108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:12.458108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:12.458108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:12.458108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:12.458108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:12.458108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:12.458108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:12.458108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:12.458108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:12.459213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:12.459213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:12.459213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:12.459213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:12.459213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:12.459213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:12.459213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:12.459213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:12.459213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:12.459213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:12.459213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:24:12.459213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:12.459213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 124/255] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:24:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:12.464898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e91b13-c865-4198-88fc-f3ff655d8238 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:12.480358 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa8cbf-a3ac-4fbc-b4e7-d4b0466102a0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:12.514493 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:12.520143 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df23e5-e3f4-4618-93d9-9b35e381f893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:12.522236 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df23e5-e3f4-4618-93d9-9b35e381f893 None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:12.523026 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df23e5-e3f4-4618-93d9-9b35e381f893 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:12.523513 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df23e5-e3f4-4618-93d9-9b35e381f893 None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0/users/222963455d3e46e5985ab76dc39a4471/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:12.526309 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df23e5-e3f4-4618-93d9-9b35e381f893 None admin] RBAC: Authorizing `identity:create_grant(project_id=d65d04e96c4b4b87be63be85bb9731a0, user_id=222963455d3e46e5985ab76dc39a4471, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:12.528065 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df23e5-e3f4-4618-93d9-9b35e381f893 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:12.554788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 132/256] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:24:12 2022] PUT /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0/users/222963455d3e46e5985ab76dc39a4471/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:24:12.595996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e91b13-c865-4198-88fc-f3ff655d8238 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:12.635651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2653df-a669-4786-8b39-9bf114096a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:12.636368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2653df-a669-4786-8b39-9bf114096a2e None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:12.636750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2653df-a669-4786-8b39-9bf114096a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:12.637086 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2653df-a669-4786-8b39-9bf114096a2e None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:12.637849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2653df-a669-4786-8b39-9bf114096a2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:12.638153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2653df-a669-4786-8b39-9bf114096a2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:12.639631 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2653df-a669-4786-8b39-9bf114096a2e None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:12.650894 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 125/257] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:12.657114 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df23e5-e3f4-4618-93d9-9b35e381f893 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:12.776204 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df23e5-e3f4-4618-93d9-9b35e381f893 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:12.813822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699d8b8-20e8-477f-a643-71e70027ed5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:12.814715 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699d8b8-20e8-477f-a643-71e70027ed5d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:12.814994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699d8b8-20e8-477f-a643-71e70027ed5d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:12.815269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699d8b8-20e8-477f-a643-71e70027ed5d None admin] PATH_INFO: `/v3/users/2330fa5347a54fb5b897b8096d0013c1` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:12.820571 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699d8b8-20e8-477f-a643-71e70027ed5d None admin] RBAC: Authorizing `identity:get_user(user_id=2330fa5347a54fb5b897b8096d0013c1)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:12.822054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699d8b8-20e8-477f-a643-71e70027ed5d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:12.824165 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 133/258] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:12 2022] GET /identity/v3/users/2330fa5347a54fb5b897b8096d0013c1 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:12.829279 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2653df-a669-4786-8b39-9bf114096a2e None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:12.856010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:12.941669 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2653df-a669-4786-8b39-9bf114096a2e None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:12.978823 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ed280-a656-43ff-aaa9-984c9d575647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:12.979473 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ed280-a656-43ff-aaa9-984c9d575647 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:12.979750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ed280-a656-43ff-aaa9-984c9d575647 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:12.980114 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ed280-a656-43ff-aaa9-984c9d575647 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:13.052006 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ed280-a656-43ff-aaa9-984c9d575647 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:13.053873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ed280-a656-43ff-aaa9-984c9d575647 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:13.075991 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:13.159242 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 126/259] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:13.165453 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699d8b8-20e8-477f-a643-71e70027ed5d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:13.230237 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:13.240823 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933a834-f707-48ba-b8f7-f0189b88d9a8 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:13.241412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933a834-f707-48ba-b8f7-f0189b88d9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:13.241891 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933a834-f707-48ba-b8f7-f0189b88d9a8 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:13.243751 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 127/260] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:24:13.257800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ea65e-f2a7-4ad5-a385-beaef760179b None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:13.258381 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ea65e-f2a7-4ad5-a385-beaef760179b None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:13.258859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ea65e-f2a7-4ad5-a385-beaef760179b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:13.276705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699d8b8-20e8-477f-a643-71e70027ed5d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:13.296801 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-644ea65e-f2a7-4ad5-a385-beaef760179b None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:13.298339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-644ea65e-f2a7-4ad5-a385-beaef760179b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:13.313556 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4defa-fe46-4a83-8054-55a1416fe79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:13.314352 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4defa-fe46-4a83-8054-55a1416fe79e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:13.314629 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4defa-fe46-4a83-8054-55a1416fe79e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:13.314901 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4defa-fe46-4a83-8054-55a1416fe79e None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:13.316113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4defa-fe46-4a83-8054-55a1416fe79e None admin] RBAC: Authorizing `identity:get_project(project_id=d65d04e96c4b4b87be63be85bb9731a0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:13.318555 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4defa-fe46-4a83-8054-55a1416fe79e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:13.321016 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 134/261] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:13 2022] GET /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:13.326941 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4defa-fe46-4a83-8054-55a1416fe79e None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:13.438025 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4defa-fe46-4a83-8054-55a1416fe79e None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:13.471673 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:13.480692 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e142054-7eff-452d-8761-e46474124910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:13.482718 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e142054-7eff-452d-8761-e46474124910 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:13.482999 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e142054-7eff-452d-8761-e46474124910 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:13.483272 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e142054-7eff-452d-8761-e46474124910 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:13.484163 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e142054-7eff-452d-8761-e46474124910 None admin] RBAC: Adding query filter params (role.id=0f929204eb7b4ea8859338ed2d3ff75b, scope.project.id=d65d04e96c4b4b87be63be85bb9731a0, user.id=2330fa5347a54fb5b897b8096d0013c1) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:13.484475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e142054-7eff-452d-8761-e46474124910 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:13.485905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e142054-7eff-452d-8761-e46474124910 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:13.500195 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 128/262] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 10:24:13.501237 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 135/263] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:24:13 2022] GET /identity/v3/role_assignments?user.id=2330fa5347a54fb5b897b8096d0013c1&scope.project.id=d65d04e96c4b4b87be63be85bb9731a0&role.id=0f929204eb7b4ea8859338ed2d3ff75b => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:13.538294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ea65e-f2a7-4ad5-a385-beaef760179b None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:13.594054 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:13.649157 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ea65e-f2a7-4ad5-a385-beaef760179b None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:13.686470 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c2c71-c33e-47bd-92ab-97ee2e9ec2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:13.687281 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c2c71-c33e-47bd-92ab-97ee2e9ec2dc None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:13.687560 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c2c71-c33e-47bd-92ab-97ee2e9ec2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:13.687852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c2c71-c33e-47bd-92ab-97ee2e9ec2dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:13.700845 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c2c71-c33e-47bd-92ab-97ee2e9ec2dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:13.709810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-624c2c71-c33e-47bd-92ab-97ee2e9ec2dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:13.709810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:13.709810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:13.709810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:13.709810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:13.709810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:13.709810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:13.709810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:13.709810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:13.709810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:13.709810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:13.709810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:13.709810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:13.709810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:13.709810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:13.709810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:13.710772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:13.710772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:13.710772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:13.710772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:13.710772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:13.710772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:13.710772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:13.710772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:13.710772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:13.710772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:13.710772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:13.710772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:13.710772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:13.710772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:13.710772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:13.710772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:13.711541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:13.711541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:13.711541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:13.711541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:13.711541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:13.711541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:13.711541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:13.711541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:13.711541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:13.711541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:13.711541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:13.711541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:13.711541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 129/264] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:24:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:13.716935 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e142054-7eff-452d-8761-e46474124910 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:13.871113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e142054-7eff-452d-8761-e46474124910 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:13.910732 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca53d64-6a34-4858-b77c-8ecc0d1b631f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:13.911419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca53d64-6a34-4858-b77c-8ecc0d1b631f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:13.911744 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca53d64-6a34-4858-b77c-8ecc0d1b631f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:13.912051 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca53d64-6a34-4858-b77c-8ecc0d1b631f None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:13.912807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca53d64-6a34-4858-b77c-8ecc0d1b631f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:13.913112 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca53d64-6a34-4858-b77c-8ecc0d1b631f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:13.914500 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca53d64-6a34-4858-b77c-8ecc0d1b631f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:13.924386 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 136/265] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:24:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:13.929997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c2c71-c33e-47bd-92ab-97ee2e9ec2dc None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:14.069932 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c2c71-c33e-47bd-92ab-97ee2e9ec2dc None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:14.081680 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:14.110762 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da47012-17ce-4bb0-82d3-cc7fb1f506d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:14.111636 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da47012-17ce-4bb0-82d3-cc7fb1f506d6 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:14.111940 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da47012-17ce-4bb0-82d3-cc7fb1f506d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:14.112212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da47012-17ce-4bb0-82d3-cc7fb1f506d6 None admin] PATH_INFO: `/v3/users/2330fa5347a54fb5b897b8096d0013c1` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:14.116733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da47012-17ce-4bb0-82d3-cc7fb1f506d6 None admin] RBAC: Authorizing `identity:get_user(user_id=2330fa5347a54fb5b897b8096d0013c1)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:14.118178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da47012-17ce-4bb0-82d3-cc7fb1f506d6 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:14.120004 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 130/266] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:13 2022] GET /identity/v3/users/2330fa5347a54fb5b897b8096d0013c1 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:14.125588 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca53d64-6a34-4858-b77c-8ecc0d1b631f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:14.220446 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:14.276566 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca53d64-6a34-4858-b77c-8ecc0d1b631f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:14.310212 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6caac-ee78-4ed6-aeff-78ac25a4970a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:14.310860 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6caac-ee78-4ed6-aeff-78ac25a4970a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:14.311142 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6caac-ee78-4ed6-aeff-78ac25a4970a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:14.311416 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6caac-ee78-4ed6-aeff-78ac25a4970a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:14.380801 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6caac-ee78-4ed6-aeff-78ac25a4970a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:14.382220 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6caac-ee78-4ed6-aeff-78ac25a4970a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:14.437075 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:14.484892 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 137/267] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:14.496973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da47012-17ce-4bb0-82d3-cc7fb1f506d6 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:14.667893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da47012-17ce-4bb0-82d3-cc7fb1f506d6 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:14.715295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3512d-8057-42f4-913b-9b82f231dfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:14.716198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3512d-8057-42f4-913b-9b82f231dfc2 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:14.716480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3512d-8057-42f4-913b-9b82f231dfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:14.716774 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3512d-8057-42f4-913b-9b82f231dfc2 None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:14.717966 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3512d-8057-42f4-913b-9b82f231dfc2 None admin] RBAC: Authorizing `identity:get_project(project_id=d65d04e96c4b4b87be63be85bb9731a0)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:14.719426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3512d-8057-42f4-913b-9b82f231dfc2 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:14.721485 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 131/268] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:14 2022] GET /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:14.727471 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6caac-ee78-4ed6-aeff-78ac25a4970a None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:14.859557 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6caac-ee78-4ed6-aeff-78ac25a4970a None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:14.915269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced35d4-fdfa-4789-88d9-eb43add1c53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:14.915961 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced35d4-fdfa-4789-88d9-eb43add1c53a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:14.916245 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced35d4-fdfa-4789-88d9-eb43add1c53a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:14.916515 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced35d4-fdfa-4789-88d9-eb43add1c53a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:14.917353 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced35d4-fdfa-4789-88d9-eb43add1c53a None admin] RBAC: Adding query filter params (role.id=f214f8dbc3354cfc81fa2c2f116980e0, scope.project.id=d65d04e96c4b4b87be63be85bb9731a0, user.id=2330fa5347a54fb5b897b8096d0013c1) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:14.917662 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced35d4-fdfa-4789-88d9-eb43add1c53a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:14.919162 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced35d4-fdfa-4789-88d9-eb43add1c53a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:14.925508 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 138/269] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:24:14 2022] GET /identity/v3/role_assignments?user.id=2330fa5347a54fb5b897b8096d0013c1&scope.project.id=d65d04e96c4b4b87be63be85bb9731a0&role.id=f214f8dbc3354cfc81fa2c2f116980e0 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:15.256278 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f8982-164c-489c-ba46-463ced06401e None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:15.256644 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f8982-164c-489c-ba46-463ced06401e None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:15.256856 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f8982-164c-489c-ba46-463ced06401e None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:15.258214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 132/270] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:15.264110 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1403f-033a-4579-8908-d0492e8f370a None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:15.264677 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1403f-033a-4579-8908-d0492e8f370a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:15.265157 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1403f-033a-4579-8908-d0492e8f370a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:15.283610 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-05d1403f-033a-4579-8908-d0492e8f370a None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:15.285155 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-05d1403f-033a-4579-8908-d0492e8f370a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:15.300236 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:15.325009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b11f82-80f9-468c-8b8f-507ae09b7ad4 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:15.325288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b11f82-80f9-468c-8b8f-507ae09b7ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:15.325717 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b11f82-80f9-468c-8b8f-507ae09b7ad4 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:15.327420 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 133/271] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:15.335265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3214570-2dfe-4370-af73-7147f65e59ba None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:15.335569 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3214570-2dfe-4370-af73-7147f65e59ba None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:15.335943 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3214570-2dfe-4370-af73-7147f65e59ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:15.362686 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-a3214570-2dfe-4370-af73-7147f65e59ba None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:15.363213 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 139/272] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:15.367804 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a3214570-2dfe-4370-af73-7147f65e59ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:15.396054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1403f-033a-4579-8908-d0492e8f370a None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:15.429042 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:15.469642 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 134/273] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:15.471033 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35633e8d-2fc7-437f-adab-210d3e9f5e24 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:15.471323 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35633e8d-2fc7-437f-adab-210d3e9f5e24 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:15.471764 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35633e8d-2fc7-437f-adab-210d3e9f5e24 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:15.473351 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 135/274] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:15.479469 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1594a6-471f-4bac-8d19-e833f3122088 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:15.479766 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1594a6-471f-4bac-8d19-e833f3122088 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:15.480085 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1594a6-471f-4bac-8d19-e833f3122088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:15.498055 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-ef1594a6-471f-4bac-8d19-e833f3122088 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:15.499235 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ef1594a6-471f-4bac-8d19-e833f3122088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:15.540978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1403f-033a-4579-8908-d0492e8f370a None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:15.570810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 136/275] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 10:24:15.575871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03478a-35ad-494a-9cf9-f8b65b041278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:15.576674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03478a-35ad-494a-9cf9-f8b65b041278 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:15.576954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03478a-35ad-494a-9cf9-f8b65b041278 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:15.577227 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03478a-35ad-494a-9cf9-f8b65b041278 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:15.580672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1594a6-471f-4bac-8d19-e833f3122088 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:15.589001 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d03478a-35ad-494a-9cf9-f8b65b041278 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:15.597171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-6d03478a-35ad-494a-9cf9-f8b65b041278 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:15.597171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:15.597171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:15.597171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:15.597171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:15.597171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:15.597171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:15.597171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:15.597171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:15.597171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:15.597171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:15.597171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:15.597171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:15.597171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:15.597171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:15.597171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:15.598063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:15.598063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:15.598063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:15.598063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:15.598063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:15.598063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:15.598063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:15.598063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:15.598063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:15.598063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:15.598063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:15.598063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:15.598063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:15.598063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:15.598063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:15.598063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:15.598846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:15.598846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:15.598846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:15.598846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:15.598846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:15.598846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:15.598846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:15.598846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:15.598846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:15.598846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:15.598846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:15.598846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:15.598846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 140/276] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:24:15.600755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03478a-35ad-494a-9cf9-f8b65b041278 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:15.610868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:15.698455 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1594a6-471f-4bac-8d19-e833f3122088 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:15.705548 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03478a-35ad-494a-9cf9-f8b65b041278 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:15.706362 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:15.737212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb395a-8d04-4190-8e1e-3d54266596bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:15.738021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb395a-8d04-4190-8e1e-3d54266596bd None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:15.738280 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb395a-8d04-4190-8e1e-3d54266596bd None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:15.738554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb395a-8d04-4190-8e1e-3d54266596bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:15.739724 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb395a-8d04-4190-8e1e-3d54266596bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:15.740198 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c02c9c-e5b0-46c0-be83-043195b04e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:15.740997 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c02c9c-e5b0-46c0-be83-043195b04e52 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:15.741275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c02c9c-e5b0-46c0-be83-043195b04e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:15.741388 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb395a-8d04-4190-8e1e-3d54266596bd None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:15.741551 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c02c9c-e5b0-46c0-be83-043195b04e52 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:15.743183 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 137/277] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:15.743803 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb395a-8d04-4190-8e1e-3d54266596bd None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:15.757621 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c02c9c-e5b0-46c0-be83-043195b04e52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:15.765482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-02c02c9c-e5b0-46c0-be83-043195b04e52 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 10:24:15.765482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:15.765482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:15.765482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:15.765482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:15.765482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:15.765482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:15.765482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:15.765482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:15.765482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:15.765482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:15.765482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:15.765482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:15.765482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:15.765482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:15.765482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:15.766381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:15.766381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:15.766381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:15.766381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:15.766381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:15.766381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:15.766381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:15.766381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:15.766381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:15.766381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:15.766381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:15.766381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:15.766381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:15.766381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:15.766381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:15.766381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:15.767167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:15.767167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:15.767167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:15.767167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:15.767167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:15.767167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:15.767167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:15.767167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:15.767167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:15.767167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:15.767167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 10:24:15.767167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:15.767167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 141/278] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:15.767853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c02c9c-e5b0-46c0-be83-043195b04e52 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:15.841061 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:15.853678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb395a-8d04-4190-8e1e-3d54266596bd None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:15.875253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c02c9c-e5b0-46c0-be83-043195b04e52 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:15.886808 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa0807-c07f-4e76-97bb-682f9cb71d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:15.887471 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa0807-c07f-4e76-97bb-682f9cb71d2d None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:15.887752 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa0807-c07f-4e76-97bb-682f9cb71d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:15.888054 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa0807-c07f-4e76-97bb-682f9cb71d2d None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:15.888788 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fa0807-c07f-4e76-97bb-682f9cb71d2d None admin] RBAC: Adding query filter params (name=member) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:15.889094 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fa0807-c07f-4e76-97bb-682f9cb71d2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:15.890476 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fa0807-c07f-4e76-97bb-682f9cb71d2d None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:15.899536 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 138/279] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:24:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:15.904138 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa0807-c07f-4e76-97bb-682f9cb71d2d None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:15.909072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f4be20-3707-472e-9612-cf909e35d7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:15.909741 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4be20-3707-472e-9612-cf909e35d7f0 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:15.910028 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4be20-3707-472e-9612-cf909e35d7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:15.910305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4be20-3707-472e-9612-cf909e35d7f0 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:15.911005 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f4be20-3707-472e-9612-cf909e35d7f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:15.913149 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f4be20-3707-472e-9612-cf909e35d7f0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:15.943846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 142/280] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jun 6 10:24:15 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:24:15.943846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabb92b-634a-4db2-8002-1fadaa1b14c4 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:15.943846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabb92b-634a-4db2-8002-1fadaa1b14c4 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:15.943846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabb92b-634a-4db2-8002-1fadaa1b14c4 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:15.943846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 143/281] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:15.943846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabb92b-634a-4db2-8002-1fadaa1b14c4 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:15.945110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:16.028673 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa0807-c07f-4e76-97bb-682f9cb71d2d None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:16.052453 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabb92b-634a-4db2-8002-1fadaa1b14c4 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:16.069057 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15086d95-b2cc-4057-878c-d1c3daf8b398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:16.069741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15086d95-b2cc-4057-878c-d1c3daf8b398 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:16.070025 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15086d95-b2cc-4057-878c-d1c3daf8b398 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:16.070302 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15086d95-b2cc-4057-878c-d1c3daf8b398 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:16.071067 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15086d95-b2cc-4057-878c-d1c3daf8b398 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:16.071371 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15086d95-b2cc-4057-878c-d1c3daf8b398 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:16.072931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15086d95-b2cc-4057-878c-d1c3daf8b398 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:16.079518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:16.083953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 139/282] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:24:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:16.085238 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff00642-d093-481b-a7f8-802b3edbb119 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:16.085612 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff00642-d093-481b-a7f8-802b3edbb119 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:16.085964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff00642-d093-481b-a7f8-802b3edbb119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:16.099175 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a31c6f-a0b2-431a-be34-89ae465e37e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:16.100133 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a31c6f-a0b2-431a-be34-89ae465e37e3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:16.100420 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a31c6f-a0b2-431a-be34-89ae465e37e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:16.100696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a31c6f-a0b2-431a-be34-89ae465e37e3 None admin] PATH_INFO: `/v3/users/7847e234ebf644a191cbe83ab53cc40d` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:16.109996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-cff00642-d093-481b-a7f8-802b3edbb119 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:16.111569 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-cff00642-d093-481b-a7f8-802b3edbb119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:16.116392 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a31c6f-a0b2-431a-be34-89ae465e37e3 None admin] RBAC: Authorizing `identity:get_user(user_id=7847e234ebf644a191cbe83ab53cc40d)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:16.117901 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a31c6f-a0b2-431a-be34-89ae465e37e3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:16.119692 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 144/283] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:15 2022] GET /identity/v3/users/7847e234ebf644a191cbe83ab53cc40d => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:16.120510 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a31c6f-a0b2-431a-be34-89ae465e37e3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:16.201820 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 140/284] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:16.202546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff00642-d093-481b-a7f8-802b3edbb119 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:16.235942 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:16.256391 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a31c6f-a0b2-431a-be34-89ae465e37e3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:16.298842 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad80349-fd23-4ee8-b670-2b813b48efcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:16.300260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad80349-fd23-4ee8-b670-2b813b48efcd None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:16.300675 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad80349-fd23-4ee8-b670-2b813b48efcd None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:16.301036 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad80349-fd23-4ee8-b670-2b813b48efcd None admin] PATH_INFO: `/v3/users/ccb58e68020044078c1a40b3fe3560b0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:16.303409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad80349-fd23-4ee8-b670-2b813b48efcd None admin] RBAC: Authorizing `identity:get_user(user_id=ccb58e68020044078c1a40b3fe3560b0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:16.304887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad80349-fd23-4ee8-b670-2b813b48efcd None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:16.306853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 145/285] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:16 2022] GET /identity/v3/users/ccb58e68020044078c1a40b3fe3560b0 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:16.307637 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad80349-fd23-4ee8-b670-2b813b48efcd None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:16.328273 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff00642-d093-481b-a7f8-802b3edbb119 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:16.366514 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:16.369097 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9d6f5-f151-41d5-a9a3-62ef933fb5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:16.370793 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9d6f5-f151-41d5-a9a3-62ef933fb5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:16.371478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9d6f5-f151-41d5-a9a3-62ef933fb5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:16.372011 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9d6f5-f151-41d5-a9a3-62ef933fb5a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:16.373149 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9d6f5-f151-41d5-a9a3-62ef933fb5a6 None admin] RBAC: Adding query filter params (role.id=6bcb06bf4aa4442ebcfebeadf1af41de, user.id=7847e234ebf644a191cbe83ab53cc40d, scope.system=all) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:16.374602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9d6f5-f151-41d5-a9a3-62ef933fb5a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:16.377061 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9d6f5-f151-41d5-a9a3-62ef933fb5a6 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:16.391316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 141/286] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Jun 6 10:24:16 2022] GET /identity/v3/role_assignments?user.id=7847e234ebf644a191cbe83ab53cc40d&scope.system=all&role.id=6bcb06bf4aa4442ebcfebeadf1af41de => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:16.392513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9d6f5-f151-41d5-a9a3-62ef933fb5a6 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:16.467350 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad80349-fd23-4ee8-b670-2b813b48efcd None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:16.512351 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:16.523433 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9d6f5-f151-41d5-a9a3-62ef933fb5a6 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:16.553486 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd0b9b-8b1b-48e8-b661-21d0d11cc588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:16.554362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd0b9b-8b1b-48e8-b661-21d0d11cc588 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:16.554590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd0b9b-8b1b-48e8-b661-21d0d11cc588 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:16.554862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd0b9b-8b1b-48e8-b661-21d0d11cc588 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:16.569236 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8dbea1-ee63-4dca-a69d-5da4229a1db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:16.570541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8dbea1-ee63-4dca-a69d-5da4229a1db7 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:16.571016 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8dbea1-ee63-4dca-a69d-5da4229a1db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:16.571496 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8dbea1-ee63-4dca-a69d-5da4229a1db7 None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:16.573182 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8dbea1-ee63-4dca-a69d-5da4229a1db7 None admin] RBAC: Authorizing `identity:get_project(project_id=d65d04e96c4b4b87be63be85bb9731a0)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:16.574880 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8dbea1-ee63-4dca-a69d-5da4229a1db7 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:16.576979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 142/287] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:16 2022] GET /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:16.585256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8dbea1-ee63-4dca-a69d-5da4229a1db7 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:16.589299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd0b9b-8b1b-48e8-b661-21d0d11cc588 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:16.608917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-3ecd0b9b-8b1b-48e8-b661-21d0d11cc588 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:16.608917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:16.608917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:16.608917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:16.608917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:16.608917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:16.608917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:16.608917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:16.608917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:16.608917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:16.608917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:16.608917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:16.608917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:16.608917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:16.608917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:16.608917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:16.609868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:16.609868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:16.609868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:16.609868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:16.609868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:16.609868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:16.609868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:16.609868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:16.609868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:16.609868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:16.609868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:16.609868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:16.609868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:16.609868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:16.609868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:16.609868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:16.610657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:16.610657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:16.610657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:16.610657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:16.610657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:16.610657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:16.610657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:16.610657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:16.610657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:16.610657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:16.610657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:16.610657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:16.610657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 146/288] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:16.616402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd0b9b-8b1b-48e8-b661-21d0d11cc588 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:16.762907 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8dbea1-ee63-4dca-a69d-5da4229a1db7 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:16.768939 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd0b9b-8b1b-48e8-b661-21d0d11cc588 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:16.788118 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:16.818302 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8b533-eab9-4731-8597-079825fce011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:16.819261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8b533-eab9-4731-8597-079825fce011 None admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:16.819482 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8b533-eab9-4731-8597-079825fce011 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:16.819967 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8b533-eab9-4731-8597-079825fce011 None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0/users/ccb58e68020044078c1a40b3fe3560b0/roles/803bb32940ee4c78ab77b3d63cd7ac0f` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:16.821966 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8b533-eab9-4731-8597-079825fce011 None admin] RBAC: Authorizing `identity:create_grant(project_id=d65d04e96c4b4b87be63be85bb9731a0, user_id=ccb58e68020044078c1a40b3fe3560b0, role_id=803bb32940ee4c78ab77b3d63cd7ac0f)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:16.823409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8b533-eab9-4731-8597-079825fce011 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:16.832194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb703a8a-d3a4-4e4b-83c2-c6a09d53be4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:16.832873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb703a8a-d3a4-4e4b-83c2-c6a09d53be4b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:16.833156 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb703a8a-d3a4-4e4b-83c2-c6a09d53be4b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:16.834154 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb703a8a-d3a4-4e4b-83c2-c6a09d53be4b None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:16.835070 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb703a8a-d3a4-4e4b-83c2-c6a09d53be4b None admin] RBAC: Adding query filter params (name=member) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:16.835384 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb703a8a-d3a4-4e4b-83c2-c6a09d53be4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:16.836894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb703a8a-d3a4-4e4b-83c2-c6a09d53be4b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:16.847189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 143/289] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:24:16 2022] PUT /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0/users/ccb58e68020044078c1a40b3fe3560b0/roles/803bb32940ee4c78ab77b3d63cd7ac0f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:24:16.853534 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 147/290] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:24:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:16.861074 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8b533-eab9-4731-8597-079825fce011 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:16.902450 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:17.022700 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8b533-eab9-4731-8597-079825fce011 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:17.070712 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda8f0a-2809-4ab7-9b0b-f86dda0c4d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:17.073619 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda8f0a-2809-4ab7-9b0b-f86dda0c4d7b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:17.074114 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda8f0a-2809-4ab7-9b0b-f86dda0c4d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:17.074590 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda8f0a-2809-4ab7-9b0b-f86dda0c4d7b None admin] PATH_INFO: `/v3/users/222963455d3e46e5985ab76dc39a4471` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:17.076077 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda8f0a-2809-4ab7-9b0b-f86dda0c4d7b None admin] RBAC: Authorizing `identity:get_user(user_id=222963455d3e46e5985ab76dc39a4471)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:17.077753 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda8f0a-2809-4ab7-9b0b-f86dda0c4d7b None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:17.079956 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 144/291] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:16 2022] GET /identity/v3/users/222963455d3e46e5985ab76dc39a4471 => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:17.085897 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb703a8a-d3a4-4e4b-83c2-c6a09d53be4b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:17.235454 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb703a8a-d3a4-4e4b-83c2-c6a09d53be4b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:17.279849 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f356cd-eb97-408e-89bf-44a11fc37d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:17.280437 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f356cd-eb97-408e-89bf-44a11fc37d31 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:17.280720 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f356cd-eb97-408e-89bf-44a11fc37d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:17.280997 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f356cd-eb97-408e-89bf-44a11fc37d31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:17.333983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:17.375326 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f356cd-eb97-408e-89bf-44a11fc37d31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:17.376825 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f356cd-eb97-408e-89bf-44a11fc37d31 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:17.526461 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 148/292] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:17.532136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda8f0a-2809-4ab7-9b0b-f86dda0c4d7b None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:17.669530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:17.690709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda8f0a-2809-4ab7-9b0b-f86dda0c4d7b None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:17.741354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ba753-8d5c-4a35-9309-290d175f82bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:17.742220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ba753-8d5c-4a35-9309-290d175f82bb None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:17.742500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ba753-8d5c-4a35-9309-290d175f82bb None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:17.742766 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ba753-8d5c-4a35-9309-290d175f82bb None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:17.743982 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ba753-8d5c-4a35-9309-290d175f82bb None admin] RBAC: Authorizing `identity:get_project(project_id=d65d04e96c4b4b87be63be85bb9731a0)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:17.745422 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ba753-8d5c-4a35-9309-290d175f82bb None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:17.747463 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 145/293] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:17 2022] GET /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:17.752919 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f356cd-eb97-408e-89bf-44a11fc37d31 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:17.868255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:17.904423 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f356cd-eb97-408e-89bf-44a11fc37d31 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:17.951322 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e2265-e452-4df4-ac64-7635f88cf9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:17.952004 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e2265-e452-4df4-ac64-7635f88cf9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:17.952286 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e2265-e452-4df4-ac64-7635f88cf9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:17.952554 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e2265-e452-4df4-ac64-7635f88cf9d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:17.953379 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e2265-e452-4df4-ac64-7635f88cf9d5 None admin] RBAC: Adding query filter params (role.id=6bcb06bf4aa4442ebcfebeadf1af41de, scope.project.id=d65d04e96c4b4b87be63be85bb9731a0, user.id=222963455d3e46e5985ab76dc39a4471) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:17.953684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e2265-e452-4df4-ac64-7635f88cf9d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:17.955105 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e2265-e452-4df4-ac64-7635f88cf9d5 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:17.964128 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 149/294] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:24:17 2022] GET /identity/v3/role_assignments?user.id=222963455d3e46e5985ab76dc39a4471&scope.project.id=d65d04e96c4b4b87be63be85bb9731a0&role.id=6bcb06bf4aa4442ebcfebeadf1af41de => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:18.717536 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f21684-f622-4962-b489-176217cb8742 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:18.718526 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f21684-f622-4962-b489-176217cb8742 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:18.719038 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f21684-f622-4962-b489-176217cb8742 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:18.720630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 146/295] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:18.726201 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc91ebe-cae5-43cd-b182-a2f23a1de699 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:18.726741 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc91ebe-cae5-43cd-b182-a2f23a1de699 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:18.727215 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc91ebe-cae5-43cd-b182-a2f23a1de699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:18.745240 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9bc91ebe-cae5-43cd-b182-a2f23a1de699 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:18.746834 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9bc91ebe-cae5-43cd-b182-a2f23a1de699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:18.820426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8773ea3-251e-4fdd-95f4-fa886e9ce033 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:18.821074 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8773ea3-251e-4fdd-95f4-fa886e9ce033 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:18.821556 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8773ea3-251e-4fdd-95f4-fa886e9ce033 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:18.823081 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 147/296] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:18.824036 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 150/297] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:18.829038 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3daa1-5de8-4572-a62f-dee580d686cc None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:18.829529 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3daa1-5de8-4572-a62f-dee580d686cc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:18.829998 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3daa1-5de8-4572-a62f-dee580d686cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:18.846290 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8773ea3-251e-4fdd-95f4-fa886e9ce033 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:18.846747 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-33a3daa1-5de8-4572-a62f-dee580d686cc None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:18.847953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-33a3daa1-5de8-4572-a62f-dee580d686cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:18.913304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:18.922655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 151/298] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:18.952102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3daa1-5de8-4572-a62f-dee580d686cc None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:18.965982 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8773ea3-251e-4fdd-95f4-fa886e9ce033 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:19.017359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb13b23-2143-430c-b00d-532f92930d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:19.018833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb13b23-2143-430c-b00d-532f92930d64 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:19.019595 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb13b23-2143-430c-b00d-532f92930d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:19.020525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb13b23-2143-430c-b00d-532f92930d64 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:19.035612 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb13b23-2143-430c-b00d-532f92930d64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:19.043948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-edb13b23-2143-430c-b00d-532f92930d64 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 10:24:19.043948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:19.043948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:19.043948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:19.043948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:19.043948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:19.043948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:19.043948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:19.043948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:19.043948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:19.043948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:19.043948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:19.043948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:19.043948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:19.043948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:19.043948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:19.045035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:19.045035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:19.045035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:19.045035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:19.045035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:19.045035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:19.045035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:19.045035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:19.045035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:19.045035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:19.045035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:19.045035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:19.045035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:19.045035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:19.045035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:19.045035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:19.045989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:19.045989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:19.045989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:19.045989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:19.045989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:19.045989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:19.045989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:19.045989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:19.045989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:19.045989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:19.045989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 10:24:19.045989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:19.051856 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 148/299] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:19.054406 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb13b23-2143-430c-b00d-532f92930d64 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:19.069776 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3daa1-5de8-4572-a62f-dee580d686cc None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:19.074441 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:19.109677 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b64a0a-e1dc-4dbe-8337-bb71b0dcfdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:19.110544 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b64a0a-e1dc-4dbe-8337-bb71b0dcfdc0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:19.110828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b64a0a-e1dc-4dbe-8337-bb71b0dcfdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:19.111107 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b64a0a-e1dc-4dbe-8337-bb71b0dcfdc0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:19.112378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b64a0a-e1dc-4dbe-8337-bb71b0dcfdc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:19.113928 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b64a0a-e1dc-4dbe-8337-bb71b0dcfdc0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:19.116836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 152/300] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:19.122053 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b64a0a-e1dc-4dbe-8337-bb71b0dcfdc0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:19.165678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb13b23-2143-430c-b00d-532f92930d64 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:19.195135 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:19.201677 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d7df3-4524-4b90-ac13-9a8765dd2819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:19.202350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d7df3-4524-4b90-ac13-9a8765dd2819 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:19.202633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d7df3-4524-4b90-ac13-9a8765dd2819 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:19.202908 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d7df3-4524-4b90-ac13-9a8765dd2819 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:19.203649 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d7df3-4524-4b90-ac13-9a8765dd2819 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:19.204003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d7df3-4524-4b90-ac13-9a8765dd2819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:19.205390 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d7df3-4524-4b90-ac13-9a8765dd2819 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:19.214035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 149/301] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:24:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:19.215477 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730e562-8dfb-4baf-87ba-94fe1b06baad None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:19.215766 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730e562-8dfb-4baf-87ba-94fe1b06baad None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:19.216066 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730e562-8dfb-4baf-87ba-94fe1b06baad None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:19.217292 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 150/302] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:19.219277 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730e562-8dfb-4baf-87ba-94fe1b06baad None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:19.244862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b64a0a-e1dc-4dbe-8337-bb71b0dcfdc0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:19.281119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a0c92-c0fd-4c06-8be7-b2dcc03a1a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:19.282340 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a0c92-c0fd-4c06-8be7-b2dcc03a1a6a None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:19.282616 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a0c92-c0fd-4c06-8be7-b2dcc03a1a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:19.282888 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a0c92-c0fd-4c06-8be7-b2dcc03a1a6a None admin] PATH_INFO: `/v3/groups` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:19.283580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a0c92-c0fd-4c06-8be7-b2dcc03a1a6a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:19.285131 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a0c92-c0fd-4c06-8be7-b2dcc03a1a6a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:19.307837 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 153/303] 10.222.0.42 () {60 vars in 1134 bytes} [Mon Jun 6 10:24:19 2022] POST /identity/v3/groups => generated 229 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:24:19.310447 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:19.317431 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1f453-7c2c-4b02-ace3-7517d1f0c29f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:19.317955 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1f453-7c2c-4b02-ace3-7517d1f0c29f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:19.318422 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1f453-7c2c-4b02-ace3-7517d1f0c29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:19.342858 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-5cc1f453-7c2c-4b02-ace3-7517d1f0c29f None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:19.344348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5cc1f453-7c2c-4b02-ace3-7517d1f0c29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:19.345784 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730e562-8dfb-4baf-87ba-94fe1b06baad None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:19.398031 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf33823-edf5-4edc-a3fe-43e91247e60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:19.400242 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf33823-edf5-4edc-a3fe-43e91247e60a None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:19.400878 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf33823-edf5-4edc-a3fe-43e91247e60a None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:19.401526 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf33823-edf5-4edc-a3fe-43e91247e60a None admin] PATH_INFO: `/v3/users/a2b8702512814f9ea8e8ba2f8d84c8a8` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:19.423912 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf33823-edf5-4edc-a3fe-43e91247e60a None admin] RBAC: Authorizing `identity:get_user(user_id=a2b8702512814f9ea8e8ba2f8d84c8a8)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:19.426530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf33823-edf5-4edc-a3fe-43e91247e60a None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:19.430123 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 151/304] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:19 2022] GET /identity/v3/users/a2b8702512814f9ea8e8ba2f8d84c8a8 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:19.435937 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf33823-edf5-4edc-a3fe-43e91247e60a None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:19.443846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 154/305] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:19.466868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1f453-7c2c-4b02-ace3-7517d1f0c29f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:19.548474 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:19.556053 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf33823-edf5-4edc-a3fe-43e91247e60a None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:19.589750 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1f453-7c2c-4b02-ace3-7517d1f0c29f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:19.603570 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de06183-89a7-44fc-8d4c-f54fe108aadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:19.604283 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de06183-89a7-44fc-8d4c-f54fe108aadd None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:19.604568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de06183-89a7-44fc-8d4c-f54fe108aadd None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:19.605165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de06183-89a7-44fc-8d4c-f54fe108aadd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:19.605681 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de06183-89a7-44fc-8d4c-f54fe108aadd None admin] RBAC: Adding query filter params (role.id=803bb32940ee4c78ab77b3d63cd7ac0f, user.id=a2b8702512814f9ea8e8ba2f8d84c8a8, scope.system=all) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:19.605994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de06183-89a7-44fc-8d4c-f54fe108aadd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:19.607429 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de06183-89a7-44fc-8d4c-f54fe108aadd None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:19.610608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:19.614069 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 152/306] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Jun 6 10:24:19 2022] GET /identity/v3/role_assignments?user.id=a2b8702512814f9ea8e8ba2f8d84c8a8&scope.system=all&role.id=803bb32940ee4c78ab77b3d63cd7ac0f => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:19.638091 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67ed32-579c-46ab-ac54-74795496265e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:19.639007 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67ed32-579c-46ab-ac54-74795496265e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:19.639284 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67ed32-579c-46ab-ac54-74795496265e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:19.639556 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67ed32-579c-46ab-ac54-74795496265e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:19.652975 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67ed32-579c-46ab-ac54-74795496265e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:19.661406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-2d67ed32-579c-46ab-ac54-74795496265e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:19.661406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:19.661406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:19.661406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:19.661406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:19.661406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:19.661406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:19.661406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:19.661406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:19.661406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:19.661406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:19.661406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:19.661406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:19.661406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:19.661406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:19.661406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:19.662362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:19.662362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:19.662362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:19.662362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:19.662362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:19.662362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:19.662362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:19.662362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:19.662362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:19.662362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:19.662362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:19.662362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:19.662362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:19.662362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:19.662362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:19.662362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:19.663159 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:19.663159 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:19.663159 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:19.663159 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:19.663159 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:19.663159 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:19.663159 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:19.663159 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:19.663159 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:19.663159 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:19.663159 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:19.663159 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:19.663758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 155/307] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:24:19.669149 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b5673-a7aa-474b-8bb0-993fb383ff49 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:19.669439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b5673-a7aa-474b-8bb0-993fb383ff49 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:19.669534 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67ed32-579c-46ab-ac54-74795496265e None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:19.669703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b5673-a7aa-474b-8bb0-993fb383ff49 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:19.671498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 153/308] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:19.676775 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba510da-ad55-41b2-aaf0-10bb1a543ff2 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:19.677689 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba510da-ad55-41b2-aaf0-10bb1a543ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:19.677854 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba510da-ad55-41b2-aaf0-10bb1a543ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:19.697330 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-dba510da-ad55-41b2-aaf0-10bb1a543ff2 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:19.698554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-dba510da-ad55-41b2-aaf0-10bb1a543ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:19.785797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 154/309] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:19.809221 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67ed32-579c-46ab-ac54-74795496265e None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:19.811448 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba510da-ad55-41b2-aaf0-10bb1a543ff2 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:19.826973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:19.856083 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e01edd-af2b-473d-8088-62f8d9a3b0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:19.857044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e01edd-af2b-473d-8088-62f8d9a3b0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:19.857935 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e01edd-af2b-473d-8088-62f8d9a3b0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:19.858384 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e01edd-af2b-473d-8088-62f8d9a3b0b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:19.859277 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e01edd-af2b-473d-8088-62f8d9a3b0b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:19.860125 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e01edd-af2b-473d-8088-62f8d9a3b0b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:19.862407 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e01edd-af2b-473d-8088-62f8d9a3b0b0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:19.879107 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 156/310] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:24:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:19.881835 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e01edd-af2b-473d-8088-62f8d9a3b0b0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:19.938674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba510da-ad55-41b2-aaf0-10bb1a543ff2 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:19.975069 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:19.983099 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d773a-6dd6-4eea-8846-665c74d34dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:19.983980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d773a-6dd6-4eea-8846-665c74d34dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:19.984261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d773a-6dd6-4eea-8846-665c74d34dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:19.984579 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d773a-6dd6-4eea-8846-665c74d34dc2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:19.998322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d773a-6dd6-4eea-8846-665c74d34dc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:20.007452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-f55d773a-6dd6-4eea-8846-665c74d34dc2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 10:24:20.007452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:20.007452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:20.007452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:20.007452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:20.007452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:20.007452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:20.007452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:20.007452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:20.007452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:20.007452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:20.007452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:20.007452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:20.007452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:20.007452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:20.007452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:20.008391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:20.008391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:20.008391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:20.008391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:20.008391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:20.008391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:20.008391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:20.008391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:20.008391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:20.008391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:20.008391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:20.008391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:20.008391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:20.008391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:20.008391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:20.008391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:20.009170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:20.009170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:20.009170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:20.009170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:20.009170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:20.009170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:20.009170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:20.009170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:20.009170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:20.009170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:20.009170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 10:24:20.009170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:20.012131 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 155/311] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:24:20.014023 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e01edd-af2b-473d-8088-62f8d9a3b0b0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:20.019922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d773a-6dd6-4eea-8846-665c74d34dc2 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:20.049090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975e179-a210-43b8-9bab-80812dc5d32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:20.050014 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975e179-a210-43b8-9bab-80812dc5d32d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:20.050295 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975e179-a210-43b8-9bab-80812dc5d32d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:20.050567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975e179-a210-43b8-9bab-80812dc5d32d None admin] PATH_INFO: `/v3/users/7847e234ebf644a191cbe83ab53cc40d` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:20.051749 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975e179-a210-43b8-9bab-80812dc5d32d None admin] RBAC: Authorizing `identity:get_user(user_id=7847e234ebf644a191cbe83ab53cc40d)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:20.054009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975e179-a210-43b8-9bab-80812dc5d32d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:20.059937 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 157/312] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:19 2022] GET /identity/v3/users/7847e234ebf644a191cbe83ab53cc40d => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:20.061400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975e179-a210-43b8-9bab-80812dc5d32d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:20.122354 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:20.138200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d773a-6dd6-4eea-8846-665c74d34dc2 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:20.178863 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd895676-c532-4325-82fa-6adbb12bcf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:20.180098 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd895676-c532-4325-82fa-6adbb12bcf9a None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:20.181094 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd895676-c532-4325-82fa-6adbb12bcf9a None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:20.181544 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd895676-c532-4325-82fa-6adbb12bcf9a None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:20.182685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd895676-c532-4325-82fa-6adbb12bcf9a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:20.183399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd895676-c532-4325-82fa-6adbb12bcf9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:20.185673 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd895676-c532-4325-82fa-6adbb12bcf9a None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:20.190195 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975e179-a210-43b8-9bab-80812dc5d32d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:20.197089 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 156/313] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:24:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:20.202229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd895676-c532-4325-82fa-6adbb12bcf9a None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:20.235517 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9ebf9-24b9-4c73-a576-add491856450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:20.236926 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9ebf9-24b9-4c73-a576-add491856450 None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:20.237672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9ebf9-24b9-4c73-a576-add491856450 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:20.238368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9ebf9-24b9-4c73-a576-add491856450 None admin] PATH_INFO: `/v3/system/users/7847e234ebf644a191cbe83ab53cc40d/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:20.241752 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9ebf9-24b9-4c73-a576-add491856450 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7847e234ebf644a191cbe83ab53cc40d, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:20.243128 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9ebf9-24b9-4c73-a576-add491856450 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:20.256461 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 158/314] 10.222.0.42 () {58 vars in 1334 bytes} [Mon Jun 6 10:24:20 2022] PUT /identity/v3/system/users/7847e234ebf644a191cbe83ab53cc40d/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:24:20.271834 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:20.330707 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd895676-c532-4325-82fa-6adbb12bcf9a None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:20.370663 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3c56f-f901-4cce-9865-24c69cd2fa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:20.371982 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3c56f-f901-4cce-9865-24c69cd2fa51 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:20.372467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3c56f-f901-4cce-9865-24c69cd2fa51 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:20.372963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3c56f-f901-4cce-9865-24c69cd2fa51 None admin] PATH_INFO: `/v3/users/ccb58e68020044078c1a40b3fe3560b0` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:20.374434 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3c56f-f901-4cce-9865-24c69cd2fa51 None admin] RBAC: Authorizing `identity:get_user(user_id=ccb58e68020044078c1a40b3fe3560b0)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:20.376231 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3c56f-f901-4cce-9865-24c69cd2fa51 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:20.378386 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 157/315] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:20 2022] GET /identity/v3/users/ccb58e68020044078c1a40b3fe3560b0 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:20.385245 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9ebf9-24b9-4c73-a576-add491856450 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:20.512614 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9ebf9-24b9-4c73-a576-add491856450 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:20.523223 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:20.551604 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7941102-715c-490e-aa8a-7dfe5a541ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:20.552384 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7941102-715c-490e-aa8a-7dfe5a541ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:20.552714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7941102-715c-490e-aa8a-7dfe5a541ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:20.552995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7941102-715c-490e-aa8a-7dfe5a541ac0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:20.627759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7941102-715c-490e-aa8a-7dfe5a541ac0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:20.629271 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7941102-715c-490e-aa8a-7dfe5a541ac0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:20.733916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 159/316] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:20.739068 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3c56f-f901-4cce-9865-24c69cd2fa51 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:20.854459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3c56f-f901-4cce-9865-24c69cd2fa51 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:20.875201 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:20.891931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed1fce-8eb4-4478-b270-015dba623fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:20.893085 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed1fce-8eb4-4478-b270-015dba623fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:20.893567 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed1fce-8eb4-4478-b270-015dba623fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:20.894056 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed1fce-8eb4-4478-b270-015dba623fa6 None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:20.895432 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ed1fce-8eb4-4478-b270-015dba623fa6 None admin] RBAC: Authorizing `identity:get_project(project_id=d65d04e96c4b4b87be63be85bb9731a0)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:20.897721 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ed1fce-8eb4-4478-b270-015dba623fa6 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:20.900236 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 158/317] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:20 2022] GET /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:20.908862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7941102-715c-490e-aa8a-7dfe5a541ac0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:20.919943 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:21.015672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7941102-715c-490e-aa8a-7dfe5a541ac0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:21.051139 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662c7c0-6541-4fba-9a85-515a049cf827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:21.052318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662c7c0-6541-4fba-9a85-515a049cf827 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:21.052856 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662c7c0-6541-4fba-9a85-515a049cf827 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:21.053197 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662c7c0-6541-4fba-9a85-515a049cf827 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:21.054038 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662c7c0-6541-4fba-9a85-515a049cf827 None admin] RBAC: Adding query filter params (role.id=803bb32940ee4c78ab77b3d63cd7ac0f, scope.project.id=d65d04e96c4b4b87be63be85bb9731a0, user.id=ccb58e68020044078c1a40b3fe3560b0) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:21.054350 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662c7c0-6541-4fba-9a85-515a049cf827 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:21.055813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662c7c0-6541-4fba-9a85-515a049cf827 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:21.062151 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 160/318] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:24:20 2022] GET /identity/v3/role_assignments?user.id=ccb58e68020044078c1a40b3fe3560b0&scope.project.id=d65d04e96c4b4b87be63be85bb9731a0&role.id=803bb32940ee4c78ab77b3d63cd7ac0f => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:21.783980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fe1e8-15c6-4266-9f89-c51a610bf31b None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:21.784349 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fe1e8-15c6-4266-9f89-c51a610bf31b None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:21.784549 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fe1e8-15c6-4266-9f89-c51a610bf31b None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:21.786050 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 159/319] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:21.791053 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a12d54-c142-441d-9159-9a57582cff71 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:21.791333 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a12d54-c142-441d-9159-9a57582cff71 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:21.791610 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a12d54-c142-441d-9159-9a57582cff71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:21.809669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c2a12d54-c142-441d-9159-9a57582cff71 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:21.810842 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c2a12d54-c142-441d-9159-9a57582cff71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:21.878013 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 161/320] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:21.901414 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fe1e8-15c6-4266-9f89-c51a610bf31b None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:21.946629 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a5368-0f6a-46a0-8f5b-f721a245fbc5 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:21.947138 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a5368-0f6a-46a0-8f5b-f721a245fbc5 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:21.947214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a5368-0f6a-46a0-8f5b-f721a245fbc5 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:21.948584 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 162/321] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:24:21.953545 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f9e19-c840-4ac5-a56a-1a1f56001449 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:21.953847 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f9e19-c840-4ac5-a56a-1a1f56001449 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:21.954118 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f9e19-c840-4ac5-a56a-1a1f56001449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:21.959369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:21.972531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-643f9e19-c840-4ac5-a56a-1a1f56001449 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:21.974310 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-643f9e19-c840-4ac5-a56a-1a1f56001449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:22.009705 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fe1e8-15c6-4266-9f89-c51a610bf31b None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:22.046975 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9081c-46b5-4cd0-9352-c82d511cb2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:22.047697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9081c-46b5-4cd0-9352-c82d511cb2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:22.048012 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9081c-46b5-4cd0-9352-c82d511cb2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:22.048289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9081c-46b5-4cd0-9352-c82d511cb2e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:22.049460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9081c-46b5-4cd0-9352-c82d511cb2e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:22.050895 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9081c-46b5-4cd0-9352-c82d511cb2e9 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:22.053124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 160/322] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:22.054931 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 163/323] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 10:24:22.060911 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9081c-46b5-4cd0-9352-c82d511cb2e9 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:22.077386 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f9e19-c840-4ac5-a56a-1a1f56001449 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:22.120382 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:22.169250 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9081c-46b5-4cd0-9352-c82d511cb2e9 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:22.197708 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f9e19-c840-4ac5-a56a-1a1f56001449 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:22.204525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80f57a-3fb2-4edd-8c9e-18ee26fd6803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:22.205193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80f57a-3fb2-4edd-8c9e-18ee26fd6803 None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:22.205479 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80f57a-3fb2-4edd-8c9e-18ee26fd6803 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:22.205756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80f57a-3fb2-4edd-8c9e-18ee26fd6803 None admin] PATH_INFO: `/v3/groups` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:22.206474 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80f57a-3fb2-4edd-8c9e-18ee26fd6803 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:22.207922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80f57a-3fb2-4edd-8c9e-18ee26fd6803 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:22.232155 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 161/324] 10.222.0.42 () {60 vars in 1134 bytes} [Mon Jun 6 10:24:22 2022] POST /identity/v3/groups => generated 226 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:24:22.232541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:22.238897 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d8e7b-2d8f-448e-a8db-6a36215c1b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:22.239842 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d8e7b-2d8f-448e-a8db-6a36215c1b20 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:22.240262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d8e7b-2d8f-448e-a8db-6a36215c1b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:22.240541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d8e7b-2d8f-448e-a8db-6a36215c1b20 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:22.253087 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d8e7b-2d8f-448e-a8db-6a36215c1b20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:22.261781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-789d8e7b-2d8f-448e-a8db-6a36215c1b20 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 10:24:22.261781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:22.261781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:22.261781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:22.261781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:22.261781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:22.261781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:22.261781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:22.261781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:22.261781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:22.261781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:22.261781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:22.261781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:22.261781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:22.261781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:22.261781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:22.262766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:22.262766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:22.262766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:22.262766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:22.262766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:22.262766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:22.262766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:22.262766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:22.262766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:22.262766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:22.262766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:22.262766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:22.262766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:22.262766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:22.262766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:22.262766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:22.263558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:22.263558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:22.263558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:22.263558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:22.263558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:22.263558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:22.263558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:22.263558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:22.263558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:22.263558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:22.263558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 10:24:22.263558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:22.265031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 164/325] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:24:22.269341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80f57a-3fb2-4edd-8c9e-18ee26fd6803 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:22.396268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80f57a-3fb2-4edd-8c9e-18ee26fd6803 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:22.414859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:22.452422 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd486d3-b0f8-4280-bcf0-06e144a6cc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:22.453104 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd486d3-b0f8-4280-bcf0-06e144a6cc34 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:22.453392 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd486d3-b0f8-4280-bcf0-06e144a6cc34 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:22.453668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd486d3-b0f8-4280-bcf0-06e144a6cc34 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:22.454419 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd486d3-b0f8-4280-bcf0-06e144a6cc34 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:22.454723 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd486d3-b0f8-4280-bcf0-06e144a6cc34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:22.456123 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd486d3-b0f8-4280-bcf0-06e144a6cc34 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:22.465626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 162/326] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:24:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:22.471398 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d8e7b-2d8f-448e-a8db-6a36215c1b20 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:22.585738 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d8e7b-2d8f-448e-a8db-6a36215c1b20 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:22.621585 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21fbdc-c354-4e6a-a1af-b961d0ea78db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:22.622467 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21fbdc-c354-4e6a-a1af-b961d0ea78db None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:22.622754 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21fbdc-c354-4e6a-a1af-b961d0ea78db None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:22.623028 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21fbdc-c354-4e6a-a1af-b961d0ea78db None admin] PATH_INFO: `/v3/users/a2b8702512814f9ea8e8ba2f8d84c8a8` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:22.624270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21fbdc-c354-4e6a-a1af-b961d0ea78db None admin] RBAC: Authorizing `identity:get_user(user_id=a2b8702512814f9ea8e8ba2f8d84c8a8)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:22.625716 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21fbdc-c354-4e6a-a1af-b961d0ea78db None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:22.627715 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 165/327] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:22 2022] GET /identity/v3/users/a2b8702512814f9ea8e8ba2f8d84c8a8 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:22.634091 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd486d3-b0f8-4280-bcf0-06e144a6cc34 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:22.709044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567e586-7620-4fdf-9a9d-33f82eb0830c None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:22.709613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567e586-7620-4fdf-9a9d-33f82eb0830c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:22.709893 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567e586-7620-4fdf-9a9d-33f82eb0830c None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:22.712293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 166/328] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:24:22.717448 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df7c2b-fb69-4cd8-a41d-982459353b48 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:22.717733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df7c2b-fb69-4cd8-a41d-982459353b48 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:22.718008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df7c2b-fb69-4cd8-a41d-982459353b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:22.739215 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-50df7c2b-fb69-4cd8-a41d-982459353b48 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:22.740797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-50df7c2b-fb69-4cd8-a41d-982459353b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:22.758200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd486d3-b0f8-4280-bcf0-06e144a6cc34 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:22.803625 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23793ba5-15e0-47ce-9192-2bed593e2407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:22.807425 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23793ba5-15e0-47ce-9192-2bed593e2407 None admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:22.807710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23793ba5-15e0-47ce-9192-2bed593e2407 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:22.808013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23793ba5-15e0-47ce-9192-2bed593e2407 None admin] PATH_INFO: `/v3/system/users/a2b8702512814f9ea8e8ba2f8d84c8a8/roles/803bb32940ee4c78ab77b3d63cd7ac0f` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:22.812572 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23793ba5-15e0-47ce-9192-2bed593e2407 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a2b8702512814f9ea8e8ba2f8d84c8a8, role_id=803bb32940ee4c78ab77b3d63cd7ac0f)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:22.814283 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23793ba5-15e0-47ce-9192-2bed593e2407 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:22.828177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:22.828824 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 163/329] 10.222.0.42 () {58 vars in 1334 bytes} [Mon Jun 6 10:24:22 2022] PUT /identity/v3/system/users/a2b8702512814f9ea8e8ba2f8d84c8a8/roles/803bb32940ee4c78ab77b3d63cd7ac0f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:24:22.858556 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 167/330] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:22.884081 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23793ba5-15e0-47ce-9192-2bed593e2407 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:22.937791 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:23.004377 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23793ba5-15e0-47ce-9192-2bed593e2407 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:23.042698 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b6c1b-b51e-4f64-9976-be93e58dd9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:23.043503 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b6c1b-b51e-4f64-9976-be93e58dd9ec None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:23.043867 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b6c1b-b51e-4f64-9976-be93e58dd9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:23.044142 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b6c1b-b51e-4f64-9976-be93e58dd9ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:23.056427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b6c1b-b51e-4f64-9976-be93e58dd9ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:23.064325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-332b6c1b-b51e-4f64-9976-be93e58dd9ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:23.064325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:23.064325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:23.064325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:23.064325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:23.064325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:23.064325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:23.064325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:23.064325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:23.064325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:23.064325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:23.064325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:23.064325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:23.064325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:23.064325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:23.064325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:23.065220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:23.065220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:23.065220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:23.065220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:23.065220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:23.065220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:23.065220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:23.065220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:23.065220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:23.065220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:23.065220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:23.065220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:23.065220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:23.065220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:23.065220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:23.065220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:23.066010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:23.066010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:23.066010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:23.066010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:23.066010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:23.066010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:23.066010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:23.066010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:23.066010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:23.066010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:23.066010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:23.066010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:23.066010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 164/331] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:23.072642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df7c2b-fb69-4cd8-a41d-982459353b48 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:23.199699 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df7c2b-fb69-4cd8-a41d-982459353b48 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:23.242714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f3b29-97fc-4ab0-8c1d-c6c62002bc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:23.243400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f3b29-97fc-4ab0-8c1d-c6c62002bc1b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:23.243685 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f3b29-97fc-4ab0-8c1d-c6c62002bc1b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:23.243982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f3b29-97fc-4ab0-8c1d-c6c62002bc1b None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:23.244736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f3b29-97fc-4ab0-8c1d-c6c62002bc1b None admin] RBAC: Adding query filter params (name=member) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:23.245043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f3b29-97fc-4ab0-8c1d-c6c62002bc1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:23.246430 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f3b29-97fc-4ab0-8c1d-c6c62002bc1b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:23.256200 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 168/332] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:24:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:23.264872 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b6c1b-b51e-4f64-9976-be93e58dd9ec None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:23.373122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b6c1b-b51e-4f64-9976-be93e58dd9ec None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:23.374724 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:23.417515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe2d9e-e87f-4e8a-991d-15e089510d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:23.418390 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe2d9e-e87f-4e8a-991d-15e089510d5d None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:23.418667 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe2d9e-e87f-4e8a-991d-15e089510d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:23.418937 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe2d9e-e87f-4e8a-991d-15e089510d5d None admin] PATH_INFO: `/v3/users/7847e234ebf644a191cbe83ab53cc40d` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:23.424539 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe2d9e-e87f-4e8a-991d-15e089510d5d None admin] RBAC: Authorizing `identity:get_user(user_id=7847e234ebf644a191cbe83ab53cc40d)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:23.425995 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe2d9e-e87f-4e8a-991d-15e089510d5d None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:23.434989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 165/333] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:23 2022] GET /identity/v3/users/7847e234ebf644a191cbe83ab53cc40d => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:23.436244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f3b29-97fc-4ab0-8c1d-c6c62002bc1b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:23.521936 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:23.585695 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f3b29-97fc-4ab0-8c1d-c6c62002bc1b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:23.635837 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645572ce-38e1-4420-bdb9-12c6a66f0f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:23.636935 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645572ce-38e1-4420-bdb9-12c6a66f0f84 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:23.637408 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645572ce-38e1-4420-bdb9-12c6a66f0f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:23.637871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645572ce-38e1-4420-bdb9-12c6a66f0f84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:23.639385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645572ce-38e1-4420-bdb9-12c6a66f0f84 None admin] RBAC: Adding query filter params (role.id=6bcb06bf4aa4442ebcfebeadf1af41de, user.id=7847e234ebf644a191cbe83ab53cc40d, scope.system=all) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:23.639941 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645572ce-38e1-4420-bdb9-12c6a66f0f84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:23.642498 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645572ce-38e1-4420-bdb9-12c6a66f0f84 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:23.662834 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 169/334] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Jun 6 10:24:23 2022] GET /identity/v3/role_assignments?user.id=7847e234ebf644a191cbe83ab53cc40d&scope.system=all&role.id=6bcb06bf4aa4442ebcfebeadf1af41de => generated 510 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:25.285887 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757dac72-9740-4a5c-b636-2c79330baa8b None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:25.286506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757dac72-9740-4a5c-b636-2c79330baa8b None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:25.286887 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757dac72-9740-4a5c-b636-2c79330baa8b None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:25.288489 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 166/335] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:25.294108 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c33da0-cf8c-410e-904d-1c03e1f238f0 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:25.294532 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c33da0-cf8c-410e-904d-1c03e1f238f0 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:25.294705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c33da0-cf8c-410e-904d-1c03e1f238f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:25.313110 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-07c33da0-cf8c-410e-904d-1c03e1f238f0 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:25.314307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-07c33da0-cf8c-410e-904d-1c03e1f238f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:25.319760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9d4d4-e16b-458a-bc01-29a363b622b6 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:25.320075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9d4d4-e16b-458a-bc01-29a363b622b6 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:25.320352 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9d4d4-e16b-458a-bc01-29a363b622b6 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:25.321844 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 167/336] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:24:25.332188 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4366fb-742c-40c7-8365-9cc00ee3203f None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:25.333548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4366fb-742c-40c7-8365-9cc00ee3203f None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:25.334215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4366fb-742c-40c7-8365-9cc00ee3203f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:25.365861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-cd4366fb-742c-40c7-8365-9cc00ee3203f None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:25.367469 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-cd4366fb-742c-40c7-8365-9cc00ee3203f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:25.462395 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 170/337] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:25.523045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 168/338] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:25.545475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c33da0-cf8c-410e-904d-1c03e1f238f0 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:25.564962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4366fb-742c-40c7-8365-9cc00ee3203f None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:25.625914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:25.764835 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4366fb-742c-40c7-8365-9cc00ee3203f None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:25.768084 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:25.775898 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c33da0-cf8c-410e-904d-1c03e1f238f0 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:25.840637 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bbe00-a051-4219-94f0-7a248b3c6ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:25.841409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bbe00-a051-4219-94f0-7a248b3c6ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:25.841690 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bbe00-a051-4219-94f0-7a248b3c6ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:25.841961 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bbe00-a051-4219-94f0-7a248b3c6ae4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:25.863291 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426bbe00-a051-4219-94f0-7a248b3c6ae4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:25.872251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-426bbe00-a051-4219-94f0-7a248b3c6ae4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:25.872251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:25.872251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:25.872251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:25.872251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:25.872251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:25.872251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:25.872251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:25.872251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:25.872251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:25.872251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:25.872251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:25.872251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:25.872251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:25.872251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:25.872251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:25.873223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:25.873223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:25.873223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:25.873223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:25.873223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:25.873223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:25.873223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:25.873223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:25.873223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:25.873223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:25.873223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:25.873223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:25.873223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:25.873223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:25.873223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:25.873223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:25.874044 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:25.874044 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:25.874044 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:25.874044 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:25.874044 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:25.874044 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:25.874044 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:25.874044 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:25.874044 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:25.874044 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:25.874044 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:25.874044 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:25.880899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 169/339] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:24:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:25.881457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5cefe-d007-43a3-8de7-7984592d5dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:25.882509 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5cefe-d007-43a3-8de7-7984592d5dae None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:25.882995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5cefe-d007-43a3-8de7-7984592d5dae None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:25.883471 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5cefe-d007-43a3-8de7-7984592d5dae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:25.891741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bbe00-a051-4219-94f0-7a248b3c6ae4 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:25.908796 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb5cefe-d007-43a3-8de7-7984592d5dae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:25.917244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-3bb5cefe-d007-43a3-8de7-7984592d5dae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:25.917244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:25.917244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:25.917244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:25.917244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:25.917244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:25.917244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:25.917244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:25.917244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:25.917244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:25.917244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:25.917244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:25.917244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:25.917244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:25.917244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:25.917244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:25.918176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:25.918176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:25.918176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:25.918176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:25.918176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:25.918176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:25.918176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:25.918176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:25.918176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:25.918176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:25.918176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:25.918176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:25.918176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:25.918176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:25.918176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:25.918176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:25.918983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:25.918983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:25.918983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:25.918983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:25.918983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:25.918983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:25.918983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:25.918983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:25.918983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:25.918983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:25.918983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:25.918983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:25.925700 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 171/340] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:25.938347 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5cefe-d007-43a3-8de7-7984592d5dae None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:26.106287 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bbe00-a051-4219-94f0-7a248b3c6ae4 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:26.123664 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5cefe-d007-43a3-8de7-7984592d5dae None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:26.126533 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:26.182244 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cdfeb4-7036-40f9-9bb9-3298515edf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:26.182911 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cdfeb4-7036-40f9-9bb9-3298515edf5f None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:26.183195 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cdfeb4-7036-40f9-9bb9-3298515edf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:26.183463 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cdfeb4-7036-40f9-9bb9-3298515edf5f None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:26.184250 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cdfeb4-7036-40f9-9bb9-3298515edf5f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:26.184551 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cdfeb4-7036-40f9-9bb9-3298515edf5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:26.185918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cdfeb4-7036-40f9-9bb9-3298515edf5f None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:26.198086 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 170/341] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:24:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:26.199765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc221c9b-11b8-4510-8a18-5212d13b14ac None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:26.200136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc221c9b-11b8-4510-8a18-5212d13b14ac None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:26.200511 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc221c9b-11b8-4510-8a18-5212d13b14ac None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:26.204749 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 171/342] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:26.207295 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6c198-a10f-46f1-b7bb-eb87d6264878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:26.208605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6c198-a10f-46f1-b7bb-eb87d6264878 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:26.208918 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6c198-a10f-46f1-b7bb-eb87d6264878 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:26.209149 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6c198-a10f-46f1-b7bb-eb87d6264878 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:26.210023 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a6c198-a10f-46f1-b7bb-eb87d6264878 None admin] RBAC: Adding query filter params (name=member) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:26.210332 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a6c198-a10f-46f1-b7bb-eb87d6264878 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:26.210440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc221c9b-11b8-4510-8a18-5212d13b14ac None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:26.211708 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a6c198-a10f-46f1-b7bb-eb87d6264878 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:26.238185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 172/343] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:24:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:26.239979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e427516-9351-4716-96e9-9ea456329e69 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:26.240344 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e427516-9351-4716-96e9-9ea456329e69 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:26.240672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e427516-9351-4716-96e9-9ea456329e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:26.246250 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:26.273235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-8e427516-9351-4716-96e9-9ea456329e69 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:26.274683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-8e427516-9351-4716-96e9-9ea456329e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:26.358822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 173/344] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:26.359597 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e427516-9351-4716-96e9-9ea456329e69 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:26.417830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc221c9b-11b8-4510-8a18-5212d13b14ac None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:26.462215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f898f81-b681-4687-b4e0-244b6d647161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:26.462867 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f898f81-b681-4687-b4e0-244b6d647161 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:26.463148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f898f81-b681-4687-b4e0-244b6d647161 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:26.463422 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f898f81-b681-4687-b4e0-244b6d647161 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:26.496063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:26.506421 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e427516-9351-4716-96e9-9ea456329e69 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:26.547828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516749fb-c9f8-4670-b280-3fd603f4f299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:26.547828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516749fb-c9f8-4670-b280-3fd603f4f299 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:26.547828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516749fb-c9f8-4670-b280-3fd603f4f299 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:26.547828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516749fb-c9f8-4670-b280-3fd603f4f299 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:26.560401 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f898f81-b681-4687-b4e0-244b6d647161 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:26.561872 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f898f81-b681-4687-b4e0-244b6d647161 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:26.589628 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:26.618881 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516749fb-c9f8-4670-b280-3fd603f4f299 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:26.620293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516749fb-c9f8-4670-b280-3fd603f4f299 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:26.693809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 172/345] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:26.695155 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cb2ec-c310-4dca-ac92-12efff88e577 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:26.695442 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cb2ec-c310-4dca-ac92-12efff88e577 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:26.695717 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cb2ec-c310-4dca-ac92-12efff88e577 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:26.697057 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 173/346] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:26.697740 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cb2ec-c310-4dca-ac92-12efff88e577 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:26.740906 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:26.763274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 174/347] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:26.768964 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516749fb-c9f8-4670-b280-3fd603f4f299 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:26.833530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cb2ec-c310-4dca-ac92-12efff88e577 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:26.872670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:26.888301 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a5fe1-20ff-49b2-9f36-10d9a440692b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:26.889155 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a5fe1-20ff-49b2-9f36-10d9a440692b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:26.889443 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a5fe1-20ff-49b2-9f36-10d9a440692b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:26.889721 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a5fe1-20ff-49b2-9f36-10d9a440692b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:26.905860 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516749fb-c9f8-4670-b280-3fd603f4f299 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:26.910597 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a5fe1-20ff-49b2-9f36-10d9a440692b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:26.919207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-158a5fe1-20ff-49b2-9f36-10d9a440692b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 10:24:26.919207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:26.919207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:26.919207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:26.919207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:26.919207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:26.919207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:26.919207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:26.919207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:26.919207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:26.919207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:26.919207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:26.919207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:26.919207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:26.919207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:26.919207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:26.920268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:26.920268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:26.920268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:26.920268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:26.920268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:26.920268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:26.920268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:26.920268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:26.920268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:26.920268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:26.920268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:26.920268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:26.920268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:26.920268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:26.920268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:26.920268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:26.922218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:26.922218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:26.922218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:26.922218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:26.922218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:26.922218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:26.922218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:26.922218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:26.922218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:26.922218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:26.922218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 10:24:26.922218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:26.922218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 174/348] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:26.922900 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cd98f-59b3-47e3-94cd-a2b634a35bc4 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:26.922900 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cd98f-59b3-47e3-94cd-a2b634a35bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:26.923262 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cd98f-59b3-47e3-94cd-a2b634a35bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:26.941019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-9f2cd98f-59b3-47e3-94cd-a2b634a35bc4 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:26.942215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-9f2cd98f-59b3-47e3-94cd-a2b634a35bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:26.953849 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48592ed0-2e77-4e33-9a9a-b573fc75e208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:26.955657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48592ed0-2e77-4e33-9a9a-b573fc75e208 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:26.956194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48592ed0-2e77-4e33-9a9a-b573fc75e208 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:26.956660 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48592ed0-2e77-4e33-9a9a-b573fc75e208 None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:26.959099 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48592ed0-2e77-4e33-9a9a-b573fc75e208 None admin] RBAC: Authorizing `identity:get_project(project_id=0efa83f60031447ab87be4f6f1af6165)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:26.961684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48592ed0-2e77-4e33-9a9a-b573fc75e208 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:26.964171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 175/349] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:26 2022] GET /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:26.965321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48592ed0-2e77-4e33-9a9a-b573fc75e208 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:27.040506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 175/350] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:27.041768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2cd98f-59b3-47e3-94cd-a2b634a35bc4 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:27.046936 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:27.112613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48592ed0-2e77-4e33-9a9a-b573fc75e208 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:27.158443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc04172-4fbb-4225-ad4f-452a11c9c386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:27.160280 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc04172-4fbb-4225-ad4f-452a11c9c386 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:27.160588 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc04172-4fbb-4225-ad4f-452a11c9c386 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:27.160869 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc04172-4fbb-4225-ad4f-452a11c9c386 None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:27.162959 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc04172-4fbb-4225-ad4f-452a11c9c386 None admin] RBAC: Authorizing `identity:get_project(project_id=0efa83f60031447ab87be4f6f1af6165)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:27.164589 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc04172-4fbb-4225-ad4f-452a11c9c386 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:27.169678 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 176/351] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:26 2022] GET /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:27.169678 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc04172-4fbb-4225-ad4f-452a11c9c386 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:27.170703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2cd98f-59b3-47e3-94cd-a2b634a35bc4 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:27.190087 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:27.234368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a96ec-6f04-4918-b4e8-aaf16115c901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:27.235519 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a96ec-6f04-4918-b4e8-aaf16115c901 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:27.236061 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a96ec-6f04-4918-b4e8-aaf16115c901 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:27.236343 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a96ec-6f04-4918-b4e8-aaf16115c901 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:27.237282 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a96ec-6f04-4918-b4e8-aaf16115c901 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:27.237884 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a96ec-6f04-4918-b4e8-aaf16115c901 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:27.240607 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a96ec-6f04-4918-b4e8-aaf16115c901 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:27.256526 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 176/352] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:24:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:27.259074 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a96ec-6f04-4918-b4e8-aaf16115c901 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:27.302824 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc04172-4fbb-4225-ad4f-452a11c9c386 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:27.338248 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:27.347974 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ef33e-8b55-455b-affb-365493b98605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:27.349782 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ef33e-8b55-455b-affb-365493b98605 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:27.350068 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ef33e-8b55-455b-affb-365493b98605 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:27.350342 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ef33e-8b55-455b-affb-365493b98605 None admin] PATH_INFO: `/v3/groups/62ddfa5186074f6eb9514f62cffa48da` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:27.357045 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ef33e-8b55-455b-affb-365493b98605 None admin] RBAC: Authorizing `identity:get_group(group_id=62ddfa5186074f6eb9514f62cffa48da)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:27.358512 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ef33e-8b55-455b-affb-365493b98605 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:27.360689 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 177/353] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jun 6 10:24:27 2022] GET /identity/v3/groups/62ddfa5186074f6eb9514f62cffa48da => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:27.362113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2143f-202d-452b-a787-4852b6cdaab7 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:27.362397 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2143f-202d-452b-a787-4852b6cdaab7 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:27.362669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2143f-202d-452b-a787-4852b6cdaab7 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:27.364158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 178/354] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:27.365056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2143f-202d-452b-a787-4852b6cdaab7 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:27.407220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a96ec-6f04-4918-b4e8-aaf16115c901 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:27.459109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43504c7-a16d-4352-a467-0d4d0c8b3f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:27.459986 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43504c7-a16d-4352-a467-0d4d0c8b3f1f None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:27.460272 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43504c7-a16d-4352-a467-0d4d0c8b3f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:27.460624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43504c7-a16d-4352-a467-0d4d0c8b3f1f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:27.473169 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43504c7-a16d-4352-a467-0d4d0c8b3f1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:27.480945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-b43504c7-a16d-4352-a467-0d4d0c8b3f1f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:27.480945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:27.480945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:27.480945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:27.480945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:27.480945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:27.480945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:27.480945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:27.480945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:27.480945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:27.480945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:27.480945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:27.480945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:27.480945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:27.480945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:27.480945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:27.481857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:27.481857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:27.481857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:27.481857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:27.481857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:27.481857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:27.481857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:27.481857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:27.481857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:27.481857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:27.481857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:27.481857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:27.481857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:27.481857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:27.481857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:27.481857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:27.482657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:27.482657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:27.482657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:27.482657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:27.482657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:27.482657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:27.482657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:27.482657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:27.482657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:27.482657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:27.482657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:27.482657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:27.482657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 177/355] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:27.483816 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43504c7-a16d-4352-a467-0d4d0c8b3f1f None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:27.516868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2143f-202d-452b-a787-4852b6cdaab7 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:27.524103 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:27.571060 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3390ea09-3d30-4162-9d14-0dce89d0f170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:27.574429 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390ea09-3d30-4162-9d14-0dce89d0f170 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:27.574713 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390ea09-3d30-4162-9d14-0dce89d0f170 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:27.574986 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390ea09-3d30-4162-9d14-0dce89d0f170 None admin] PATH_INFO: `/v3/groups/62ddfa5186074f6eb9514f62cffa48da` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:27.577169 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3390ea09-3d30-4162-9d14-0dce89d0f170 None admin] RBAC: Authorizing `identity:get_group(group_id=62ddfa5186074f6eb9514f62cffa48da)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:27.578635 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3390ea09-3d30-4162-9d14-0dce89d0f170 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:27.580322 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 179/356] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jun 6 10:24:27 2022] GET /identity/v3/groups/62ddfa5186074f6eb9514f62cffa48da => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:27.581089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3390ea09-3d30-4162-9d14-0dce89d0f170 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:27.627669 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43504c7-a16d-4352-a467-0d4d0c8b3f1f None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:27.652078 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:27.667349 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668abb2-7b08-49dc-bdb8-5961458d97dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:27.668250 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668abb2-7b08-49dc-bdb8-5961458d97dc None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:27.668530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668abb2-7b08-49dc-bdb8-5961458d97dc None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:27.668806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668abb2-7b08-49dc-bdb8-5961458d97dc None admin] PATH_INFO: `/v3/users/a2b8702512814f9ea8e8ba2f8d84c8a8` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:27.670006 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668abb2-7b08-49dc-bdb8-5961458d97dc None admin] RBAC: Authorizing `identity:get_user(user_id=a2b8702512814f9ea8e8ba2f8d84c8a8)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:27.671447 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668abb2-7b08-49dc-bdb8-5961458d97dc None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:27.673125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 178/357] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:24:27 2022] GET /identity/v3/users/a2b8702512814f9ea8e8ba2f8d84c8a8 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:27.674282 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011898c-cc87-4c92-beab-f5d5ef0cf646 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:27.674565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011898c-cc87-4c92-beab-f5d5ef0cf646 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:27.674840 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011898c-cc87-4c92-beab-f5d5ef0cf646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:27.692625 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-5011898c-cc87-4c92-beab-f5d5ef0cf646 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:27.693822 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-5011898c-cc87-4c92-beab-f5d5ef0cf646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:27.702139 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3390ea09-3d30-4162-9d14-0dce89d0f170 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:27.742691 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ce8fe-29fb-4ae5-ab15-da16ce663121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:27.743271 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ce8fe-29fb-4ae5-ab15-da16ce663121 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:27.743548 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ce8fe-29fb-4ae5-ab15-da16ce663121 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:27.743850 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ce8fe-29fb-4ae5-ab15-da16ce663121 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:27.744686 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ce8fe-29fb-4ae5-ab15-da16ce663121 None admin] RBAC: Adding query filter params (group.id=62ddfa5186074f6eb9514f62cffa48da, role.id=f214f8dbc3354cfc81fa2c2f116980e0, scope.project.id=0efa83f60031447ab87be4f6f1af6165) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:27.744995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ce8fe-29fb-4ae5-ab15-da16ce663121 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:27.746440 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ce8fe-29fb-4ae5-ab15-da16ce663121 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:27.753484 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 180/358] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jun 6 10:24:27 2022] GET /identity/v3/role_assignments?group.id=62ddfa5186074f6eb9514f62cffa48da&scope.project.id=0efa83f60031447ab87be4f6f1af6165&role.id=f214f8dbc3354cfc81fa2c2f116980e0 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:27.754288 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ce8fe-29fb-4ae5-ab15-da16ce663121 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:27.778221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 179/359] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:27.779356 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011898c-cc87-4c92-beab-f5d5ef0cf646 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:27.824908 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:27.911901 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ce8fe-29fb-4ae5-ab15-da16ce663121 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:27.925181 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011898c-cc87-4c92-beab-f5d5ef0cf646 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:27.947259 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:27.961415 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec289e32-a022-4616-9df2-9b13809c31ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:27.962098 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec289e32-a022-4616-9df2-9b13809c31ad None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:27.962379 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec289e32-a022-4616-9df2-9b13809c31ad None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:27.962656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec289e32-a022-4616-9df2-9b13809c31ad None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:27.963413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec289e32-a022-4616-9df2-9b13809c31ad None admin] RBAC: Adding query filter params (name=member) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:27.963721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec289e32-a022-4616-9df2-9b13809c31ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:27.965165 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec289e32-a022-4616-9df2-9b13809c31ad None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:27.975099 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd27bd-2867-4e1d-8b4c-6c6fd5ff1f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:27.976114 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd27bd-2867-4e1d-8b4c-6c6fd5ff1f49 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:27.976591 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd27bd-2867-4e1d-8b4c-6c6fd5ff1f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:27.976989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 181/360] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:24:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:27.977246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd27bd-2867-4e1d-8b4c-6c6fd5ff1f49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:27.978267 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd27bd-2867-4e1d-8b4c-6c6fd5ff1f49 None admin] RBAC: Adding query filter params (group.id=62ddfa5186074f6eb9514f62cffa48da, role.id=6bcb06bf4aa4442ebcfebeadf1af41de, scope.project.id=0efa83f60031447ab87be4f6f1af6165) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:27.978384 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72226d-0be0-4bec-a11b-2b497cbd8ebd None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:27.978591 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72226d-0be0-4bec-a11b-2b497cbd8ebd None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:27.978867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72226d-0be0-4bec-a11b-2b497cbd8ebd None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:27.979232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd27bd-2867-4e1d-8b4c-6c6fd5ff1f49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:27.980401 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 182/361] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:27.981304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72226d-0be0-4bec-a11b-2b497cbd8ebd None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:27.981578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd27bd-2867-4e1d-8b4c-6c6fd5ff1f49 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:27.988108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 180/362] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jun 6 10:24:27 2022] GET /identity/v3/role_assignments?group.id=62ddfa5186074f6eb9514f62cffa48da&scope.project.id=0efa83f60031447ab87be4f6f1af6165&role.id=6bcb06bf4aa4442ebcfebeadf1af41de => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:27.989306 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd27bd-2867-4e1d-8b4c-6c6fd5ff1f49 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:28.121198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:28.124574 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72226d-0be0-4bec-a11b-2b497cbd8ebd None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:28.130190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd27bd-2867-4e1d-8b4c-6c6fd5ff1f49 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:28.178312 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f6140-783e-4062-b142-83398fe36ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:28.179101 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e658f-5b1d-4de1-9d34-e2ccde7015d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:28.179657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f6140-783e-4062-b142-83398fe36ceb None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:28.179872 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e658f-5b1d-4de1-9d34-e2ccde7015d6 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:28.180133 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e658f-5b1d-4de1-9d34-e2ccde7015d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:28.180406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e658f-5b1d-4de1-9d34-e2ccde7015d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:28.180697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f6140-783e-4062-b142-83398fe36ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:28.181151 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f6140-783e-4062-b142-83398fe36ceb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:28.181262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e658f-5b1d-4de1-9d34-e2ccde7015d6 None admin] RBAC: Adding query filter params (role.id=803bb32940ee4c78ab77b3d63cd7ac0f, user.id=a2b8702512814f9ea8e8ba2f8d84c8a8, scope.system=all) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:28.181561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e658f-5b1d-4de1-9d34-e2ccde7015d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:28.183034 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e658f-5b1d-4de1-9d34-e2ccde7015d6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:28.195963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f6140-783e-4062-b142-83398fe36ceb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:28.203195 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 183/363] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Jun 6 10:24:27 2022] GET /identity/v3/role_assignments?user.id=a2b8702512814f9ea8e8ba2f8d84c8a8&scope.system=all&role.id=803bb32940ee4c78ab77b3d63cd7ac0f => generated 510 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:28.204154 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e658f-5b1d-4de1-9d34-e2ccde7015d6 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:28.206308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-601f6140-783e-4062-b142-83398fe36ceb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:28.206308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:28.206308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:28.206308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:28.206308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:28.206308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:28.206308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:28.206308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:28.206308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:28.206308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:28.206308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:28.206308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:28.206308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:28.206308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:28.206308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:28.206308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:28.207204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:28.207204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:28.207204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:28.207204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:28.207204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:28.207204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:28.207204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:28.207204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:28.207204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:28.207204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:28.207204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:28.207204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:28.207204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:28.207204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:28.207204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:28.207204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:28.213013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:28.213013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:28.213013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:28.213013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:28.213013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:28.213013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:28.213013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:28.213013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:28.213013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:28.213013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:28.213013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:28.213013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:28.213013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 181/364] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:24:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:28.213013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879eb08-3064-45b9-b179-26c848f6fcdd None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:28.213787 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879eb08-3064-45b9-b179-26c848f6fcdd None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:28.213787 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879eb08-3064-45b9-b179-26c848f6fcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:28.251009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-7879eb08-3064-45b9-b179-26c848f6fcdd None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:28.252292 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-7879eb08-3064-45b9-b179-26c848f6fcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:28.309895 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:28.391200 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e658f-5b1d-4de1-9d34-e2ccde7015d6 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:28.434265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56648eb9-a76a-4755-8121-745828e5d985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:28.434547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 182/365] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:28.435075 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56648eb9-a76a-4755-8121-745828e5d985 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:28.435371 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879eb08-3064-45b9-b179-26c848f6fcdd None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:28.435476 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56648eb9-a76a-4755-8121-745828e5d985 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:28.435809 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56648eb9-a76a-4755-8121-745828e5d985 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:28.534390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56648eb9-a76a-4755-8121-745828e5d985 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:28.536281 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56648eb9-a76a-4755-8121-745828e5d985 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:28.590792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:28.598372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:28.618274 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879eb08-3064-45b9-b179-26c848f6fcdd None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:28.665872 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 184/366] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:28.667044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56648eb9-a76a-4755-8121-745828e5d985 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:28.667438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e786c-2b8e-4d94-b401-ffde204b04a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:28.671805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e786c-2b8e-4d94-b401-ffde204b04a6 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:28.671805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e786c-2b8e-4d94-b401-ffde204b04a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:28.671805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e786c-2b8e-4d94-b401-ffde204b04a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:28.671805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e786c-2b8e-4d94-b401-ffde204b04a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:28.671805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e786c-2b8e-4d94-b401-ffde204b04a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:28.671805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e786c-2b8e-4d94-b401-ffde204b04a6 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:28.682603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 183/367] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:24:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:28.683445 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e786c-2b8e-4d94-b401-ffde204b04a6 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:28.794261 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56648eb9-a76a-4755-8121-745828e5d985 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:28.818438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e786c-2b8e-4d94-b401-ffde204b04a6 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:28.833961 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3fb8e-897f-4ea0-86c5-9df16e1f40e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:28.835089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3fb8e-897f-4ea0-86c5-9df16e1f40e9 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:28.835572 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3fb8e-897f-4ea0-86c5-9df16e1f40e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:28.836073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3fb8e-897f-4ea0-86c5-9df16e1f40e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:28.849719 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3fb8e-897f-4ea0-86c5-9df16e1f40e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:28.858786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-7aa3fb8e-897f-4ea0-86c5-9df16e1f40e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:24:28.858786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:28.858786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:28.858786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:28.858786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:28.858786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:28.858786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:28.858786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:28.858786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:28.858786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:28.858786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:28.858786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:28.858786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:28.858786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:28.858786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:28.858786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:28.859696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:28.859696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:28.859696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:28.859696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:28.859696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:28.859696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:28.859696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:28.859696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:28.859696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:28.859696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:28.859696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:28.859696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:28.859696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:28.859696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:28.859696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:28.859696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:28.860841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:28.860841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:28.860841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:28.860841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:28.860841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:28.860841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:28.860841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:28.860841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:28.860841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:28.860841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:28.860841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:24:28.860841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:28.861484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc3029-6ffa-4d95-8d98-47587770b843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:28.861484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc3029-6ffa-4d95-8d98-47587770b843 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:28.861484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc3029-6ffa-4d95-8d98-47587770b843 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:28.861484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc3029-6ffa-4d95-8d98-47587770b843 None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:28.862279 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc3029-6ffa-4d95-8d98-47587770b843 None admin] RBAC: Authorizing `identity:get_project(project_id=d65d04e96c4b4b87be63be85bb9731a0)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:28.863731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc3029-6ffa-4d95-8d98-47587770b843 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:28.864700 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 185/368] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:24:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:28.865692 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3fb8e-897f-4ea0-86c5-9df16e1f40e9 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:28.866056 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 184/369] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:28 2022] GET /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:28.871597 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc3029-6ffa-4d95-8d98-47587770b843 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:28.875481 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:28.897047 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:28.999683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3fb8e-897f-4ea0-86c5-9df16e1f40e9 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:28.999904 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc3029-6ffa-4d95-8d98-47587770b843 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:29.045161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de154e-24f1-4877-8cf2-14b3efa5e61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:29.045675 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99110e3-1865-4444-ac0f-cebd358ae3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:29.045875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de154e-24f1-4877-8cf2-14b3efa5e61d None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:29.046110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de154e-24f1-4877-8cf2-14b3efa5e61d None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:29.046384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de154e-24f1-4877-8cf2-14b3efa5e61d None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:29.047133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de154e-24f1-4877-8cf2-14b3efa5e61d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:29.047316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99110e3-1865-4444-ac0f-cebd358ae3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:29.047418 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de154e-24f1-4877-8cf2-14b3efa5e61d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:29.047919 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99110e3-1865-4444-ac0f-cebd358ae3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:29.048388 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99110e3-1865-4444-ac0f-cebd358ae3d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:29.048829 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de154e-24f1-4877-8cf2-14b3efa5e61d None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:29.059727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 185/370] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:29.060585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de154e-24f1-4877-8cf2-14b3efa5e61d None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:29.128329 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99110e3-1865-4444-ac0f-cebd358ae3d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:29.130602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99110e3-1865-4444-ac0f-cebd358ae3d7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:29.133830 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:29.147201 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:29.170785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de154e-24f1-4877-8cf2-14b3efa5e61d None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:29.206611 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f4e84-9063-4660-9073-8cf82dd5225f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:29.207465 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f4e84-9063-4660-9073-8cf82dd5225f None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:29.207800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f4e84-9063-4660-9073-8cf82dd5225f None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:29.208085 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f4e84-9063-4660-9073-8cf82dd5225f None admin] PATH_INFO: `/v3/groups/62ddfa5186074f6eb9514f62cffa48da` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:29.209329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f4e84-9063-4660-9073-8cf82dd5225f None admin] RBAC: Authorizing `identity:get_group(group_id=62ddfa5186074f6eb9514f62cffa48da)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:29.210777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f4e84-9063-4660-9073-8cf82dd5225f None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:29.214637 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 186/371] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jun 6 10:24:29 2022] GET /identity/v3/groups/62ddfa5186074f6eb9514f62cffa48da => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:29.214637 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f4e84-9063-4660-9073-8cf82dd5225f None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:29.264926 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 186/372] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:29.266089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99110e3-1865-4444-ac0f-cebd358ae3d7 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:29.349510 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f4e84-9063-4660-9073-8cf82dd5225f None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:29.394125 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99110e3-1865-4444-ac0f-cebd358ae3d7 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:29.397600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:29.400752 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d016e-7091-4722-8d9d-0e2f71027f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:29.401685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d016e-7091-4722-8d9d-0e2f71027f73 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:29.402114 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d016e-7091-4722-8d9d-0e2f71027f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:29.402825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d016e-7091-4722-8d9d-0e2f71027f73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:29.426209 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:29.443489 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcdbf7-6851-46ac-85d6-ea5534a5e21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:29.444883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcdbf7-6851-46ac-85d6-ea5534a5e21d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:29.445332 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcdbf7-6851-46ac-85d6-ea5534a5e21d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:29.446100 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcdbf7-6851-46ac-85d6-ea5534a5e21d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:29.447252 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcdbf7-6851-46ac-85d6-ea5534a5e21d None admin] RBAC: Adding query filter params (group.id=62ddfa5186074f6eb9514f62cffa48da, role.id=6bcb06bf4aa4442ebcfebeadf1af41de, scope.project.id=d65d04e96c4b4b87be63be85bb9731a0) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:29.447723 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcdbf7-6851-46ac-85d6-ea5534a5e21d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:29.449858 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcdbf7-6851-46ac-85d6-ea5534a5e21d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:29.457599 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 187/373] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jun 6 10:24:29 2022] GET /identity/v3/role_assignments?group.id=62ddfa5186074f6eb9514f62cffa48da&scope.project.id=d65d04e96c4b4b87be63be85bb9731a0&role.id=6bcb06bf4aa4442ebcfebeadf1af41de => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:29.458638 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcdbf7-6851-46ac-85d6-ea5534a5e21d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:29.490920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d016e-7091-4722-8d9d-0e2f71027f73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:29.492573 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d016e-7091-4722-8d9d-0e2f71027f73 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:29.571470 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcdbf7-6851-46ac-85d6-ea5534a5e21d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:29.609679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2906d1e-a20d-4d4a-9506-621fa57e8bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:29.610551 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2906d1e-a20d-4d4a-9506-621fa57e8bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:29.610836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2906d1e-a20d-4d4a-9506-621fa57e8bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:29.611115 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2906d1e-a20d-4d4a-9506-621fa57e8bd6 None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:29.613352 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2906d1e-a20d-4d4a-9506-621fa57e8bd6 None admin] RBAC: Authorizing `identity:get_project(project_id=d65d04e96c4b4b87be63be85bb9731a0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:29.615388 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2906d1e-a20d-4d4a-9506-621fa57e8bd6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:29.617168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 188/374] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:29 2022] GET /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:29.617969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 187/375] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:29.622661 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2906d1e-a20d-4d4a-9506-621fa57e8bd6 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:29.623961 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d016e-7091-4722-8d9d-0e2f71027f73 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:29.689533 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:29.709890 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:29.797873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d016e-7091-4722-8d9d-0e2f71027f73 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:29.799368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2906d1e-a20d-4d4a-9506-621fa57e8bd6 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:29.835479 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa72da-2a70-4063-b430-1598ee1091d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:29.836658 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa72da-2a70-4063-b430-1598ee1091d6 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:29.837131 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa72da-2a70-4063-b430-1598ee1091d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:29.837627 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa72da-2a70-4063-b430-1598ee1091d6 None admin] PATH_INFO: `/v3/projects/863195bccbf14e04a9a61a90f52ba9f1` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:29.842405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa72da-2a70-4063-b430-1598ee1091d6 None admin] RBAC: Authorizing `identity:get_project(project_id=863195bccbf14e04a9a61a90f52ba9f1)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:29.843383 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5015b04b-2173-4de2-b19f-4d98f6d68730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:29.844282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015b04b-2173-4de2-b19f-4d98f6d68730 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:29.844551 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa72da-2a70-4063-b430-1598ee1091d6 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:29.844651 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015b04b-2173-4de2-b19f-4d98f6d68730 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:29.844826 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015b04b-2173-4de2-b19f-4d98f6d68730 None admin] PATH_INFO: `/v3/groups/62ddfa5186074f6eb9514f62cffa48da` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:29.846011 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5015b04b-2173-4de2-b19f-4d98f6d68730 None admin] RBAC: Authorizing `identity:get_group(group_id=62ddfa5186074f6eb9514f62cffa48da)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:29.847267 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 188/376] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:29 2022] GET /identity/v3/projects/863195bccbf14e04a9a61a90f52ba9f1 => generated 344 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:29.849777 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5015b04b-2173-4de2-b19f-4d98f6d68730 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:29.849777 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 189/377] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jun 6 10:24:29 2022] GET /identity/v3/groups/62ddfa5186074f6eb9514f62cffa48da => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:29.855049 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa72da-2a70-4063-b430-1598ee1091d6 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:29.856250 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5015b04b-2173-4de2-b19f-4d98f6d68730 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:29.981774 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5015b04b-2173-4de2-b19f-4d98f6d68730 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:29.984157 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:30.019491 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cdb17-44cd-40ac-adb7-c459a0469d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:30.020209 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cdb17-44cd-40ac-adb7-c459a0469d0e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:30.020499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cdb17-44cd-40ac-adb7-c459a0469d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:30.020783 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cdb17-44cd-40ac-adb7-c459a0469d0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:30.021619 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cdb17-44cd-40ac-adb7-c459a0469d0e None admin] RBAC: Adding query filter params (group.id=62ddfa5186074f6eb9514f62cffa48da, role.id=f214f8dbc3354cfc81fa2c2f116980e0, scope.project.id=d65d04e96c4b4b87be63be85bb9731a0) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:30.021928 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cdb17-44cd-40ac-adb7-c459a0469d0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:30.023366 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cdb17-44cd-40ac-adb7-c459a0469d0e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:30.030884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 190/378] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jun 6 10:24:29 2022] GET /identity/v3/role_assignments?group.id=62ddfa5186074f6eb9514f62cffa48da&scope.project.id=d65d04e96c4b4b87be63be85bb9731a0&role.id=f214f8dbc3354cfc81fa2c2f116980e0 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:30.041989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa72da-2a70-4063-b430-1598ee1091d6 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:30.069965 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:30.097741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d37b3-2c0f-46a5-ac2d-f2ae4e293244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:30.098592 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d37b3-2c0f-46a5-ac2d-f2ae4e293244 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:30.098874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d37b3-2c0f-46a5-ac2d-f2ae4e293244 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:30.099152 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d37b3-2c0f-46a5-ac2d-f2ae4e293244 None admin] PATH_INFO: `/v3/groups/83a537bba06a44fbb6d102d17532495d` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:30.107528 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d37b3-2c0f-46a5-ac2d-f2ae4e293244 None admin] RBAC: Authorizing `identity:get_group(group_id=83a537bba06a44fbb6d102d17532495d)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:30.109028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d37b3-2c0f-46a5-ac2d-f2ae4e293244 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:30.111060 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 189/379] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jun 6 10:24:29 2022] GET /identity/v3/groups/83a537bba06a44fbb6d102d17532495d => generated 229 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:30.117511 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cdb17-44cd-40ac-adb7-c459a0469d0e None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:30.258052 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cdb17-44cd-40ac-adb7-c459a0469d0e None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:30.302528 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb6f3d-8347-4248-b4dc-03ebe7ce51c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:30.303311 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb6f3d-8347-4248-b4dc-03ebe7ce51c5 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:30.303572 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb6f3d-8347-4248-b4dc-03ebe7ce51c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:30.303873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb6f3d-8347-4248-b4dc-03ebe7ce51c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:30.304732 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb6f3d-8347-4248-b4dc-03ebe7ce51c5 None admin] RBAC: Adding query filter params (group.id=83a537bba06a44fbb6d102d17532495d, role.id=0f929204eb7b4ea8859338ed2d3ff75b, scope.project.id=863195bccbf14e04a9a61a90f52ba9f1) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:30.305040 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb6f3d-8347-4248-b4dc-03ebe7ce51c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:30.306477 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb6f3d-8347-4248-b4dc-03ebe7ce51c5 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:30.312521 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 191/380] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jun 6 10:24:30 2022] GET /identity/v3/role_assignments?group.id=83a537bba06a44fbb6d102d17532495d&scope.project.id=863195bccbf14e04a9a61a90f52ba9f1&role.id=0f929204eb7b4ea8859338ed2d3ff75b => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:30.684090 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a60864-53c9-454f-8d15-44141803677c None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:30.684570 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a60864-53c9-454f-8d15-44141803677c None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:30.684645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a60864-53c9-454f-8d15-44141803677c None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:30.686018 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 190/381] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:30.691242 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be30845-c343-4e62-8537-c607c7458ad4 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:30.691642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be30845-c343-4e62-8537-c607c7458ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:30.691829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be30845-c343-4e62-8537-c607c7458ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:30.716028 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-5be30845-c343-4e62-8537-c607c7458ad4 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:30.717360 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5be30845-c343-4e62-8537-c607c7458ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:30.759244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:30.792291 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f51ace-ccd6-4a7d-9b22-fa9214e5b4e6 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:30.792606 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f51ace-ccd6-4a7d-9b22-fa9214e5b4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:30.792842 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f51ace-ccd6-4a7d-9b22-fa9214e5b4e6 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:30.794435 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 191/382] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:30.799934 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b9533-8acc-4423-9d59-14419ef619fd None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:30.800217 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b9533-8acc-4423-9d59-14419ef619fd None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:30.800493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b9533-8acc-4423-9d59-14419ef619fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:30.818057 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 192/383] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:30.820189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-fb2b9533-8acc-4423-9d59-14419ef619fd None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:30.821384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-fb2b9533-8acc-4423-9d59-14419ef619fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:30.853203 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be30845-c343-4e62-8537-c607c7458ad4 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:30.917227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 192/384] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 10:24:30.957005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b9533-8acc-4423-9d59-14419ef619fd None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:31.027387 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be30845-c343-4e62-8537-c607c7458ad4 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:31.079716 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3c667-a72a-435d-9032-dba47e46212b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:31.080909 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3c667-a72a-435d-9032-dba47e46212b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:31.081382 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3c667-a72a-435d-9032-dba47e46212b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:31.084530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3c667-a72a-435d-9032-dba47e46212b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:31.103842 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a3c667-a72a-435d-9032-dba47e46212b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:31.114009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-35a3c667-a72a-435d-9032-dba47e46212b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:31.114009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:31.114009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:31.114009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:31.114009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:31.114009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:31.114009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:31.114009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:31.114009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:31.114009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:31.114009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:31.114009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:31.114009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:31.114009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:31.114009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:31.114009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:31.115900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:31.115900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:31.115900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:31.115900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:31.115900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:31.115900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:31.115900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:31.115900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:31.115900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:31.115900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:31.115900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:31.115900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:31.115900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:31.115900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:31.115900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:31.115900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:31.116703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:31.116703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:31.116703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:31.116703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:31.116703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:31.116703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:31.116703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:31.116703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:31.116703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:31.116703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:31.116703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:31.116703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:31.119854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 193/385] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:24:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:24:31.125214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3c667-a72a-435d-9032-dba47e46212b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:31.167516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:31.175727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b9533-8acc-4423-9d59-14419ef619fd None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:31.199442 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:31.249094 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01115e-bf40-4877-925b-fbad73956d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:31.250246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01115e-bf40-4877-925b-fbad73956d97 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:31.250763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01115e-bf40-4877-925b-fbad73956d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:31.252078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01115e-bf40-4877-925b-fbad73956d97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:31.269669 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01115e-bf40-4877-925b-fbad73956d97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:31.280970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-fe01115e-bf40-4877-925b-fbad73956d97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:31.280970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:31.280970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:31.280970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:31.280970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:31.280970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:31.280970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:31.280970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:31.280970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:31.280970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:31.280970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:31.280970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:31.280970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:31.280970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:31.280970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:31.280970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:31.281936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:31.281936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:31.281936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:31.281936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:31.281936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:31.281936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:31.281936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:31.281936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:31.281936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:31.281936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:31.281936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:31.281936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:31.281936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:31.281936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:31.281936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:31.281936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:31.282855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:31.282855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:31.282855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:31.282855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:31.282855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:31.282855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:31.282855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:31.282855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:31.282855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:31.282855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:31.282855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:31.282855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:31.284044 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 193/386] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:31.289860 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01115e-bf40-4877-925b-fbad73956d97 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:31.340085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3c667-a72a-435d-9032-dba47e46212b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:31.414084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637e4b8-fc4d-4929-9035-f42581121a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:31.415724 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637e4b8-fc4d-4929-9035-f42581121a4d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:31.417391 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637e4b8-fc4d-4929-9035-f42581121a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:31.417822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637e4b8-fc4d-4929-9035-f42581121a4d None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:31.419560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2637e4b8-fc4d-4929-9035-f42581121a4d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:31.420956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2637e4b8-fc4d-4929-9035-f42581121a4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:31.423913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2637e4b8-fc4d-4929-9035-f42581121a4d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:31.442028 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 194/387] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:24:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:31.467951 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637e4b8-fc4d-4929-9035-f42581121a4d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:31.500740 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01115e-bf40-4877-925b-fbad73956d97 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:31.567928 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061dcdc-ee41-4851-93c5-16d369f885af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:31.569259 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061dcdc-ee41-4851-93c5-16d369f885af None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:31.569687 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061dcdc-ee41-4851-93c5-16d369f885af None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:31.570130 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061dcdc-ee41-4851-93c5-16d369f885af None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:31.571026 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061dcdc-ee41-4851-93c5-16d369f885af None admin] RBAC: Adding query filter params (name=member) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:31.571503 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061dcdc-ee41-4851-93c5-16d369f885af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:31.573308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061dcdc-ee41-4851-93c5-16d369f885af None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:31.583468 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 194/388] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:24:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:31.592733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061dcdc-ee41-4851-93c5-16d369f885af None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:31.633093 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:31.638094 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:31.669002 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637e4b8-fc4d-4929-9035-f42581121a4d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:31.718822 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061dcdc-ee41-4851-93c5-16d369f885af None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:31.737072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3178e-12aa-40bc-8d55-8522f8d071b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:31.737965 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3178e-12aa-40bc-8d55-8522f8d071b2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:31.738253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3178e-12aa-40bc-8d55-8522f8d071b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:31.738534 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3178e-12aa-40bc-8d55-8522f8d071b2 None admin] PATH_INFO: `/v3/groups/62ddfa5186074f6eb9514f62cffa48da` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:31.744548 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f3178e-12aa-40bc-8d55-8522f8d071b2 None admin] RBAC: Authorizing `identity:get_group(group_id=62ddfa5186074f6eb9514f62cffa48da)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:31.746029 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f3178e-12aa-40bc-8d55-8522f8d071b2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:31.748046 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 195/389] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jun 6 10:24:31 2022] GET /identity/v3/groups/62ddfa5186074f6eb9514f62cffa48da => generated 226 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:31.759664 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3178e-12aa-40bc-8d55-8522f8d071b2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:31.761915 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0be328-6afd-4988-be1d-4fcb80ab2121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:31.762801 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0be328-6afd-4988-be1d-4fcb80ab2121 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:31.763078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0be328-6afd-4988-be1d-4fcb80ab2121 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:31.763350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0be328-6afd-4988-be1d-4fcb80ab2121 None admin] PATH_INFO: `/v3/groups/62ddfa5186074f6eb9514f62cffa48da` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:31.764536 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0be328-6afd-4988-be1d-4fcb80ab2121 None admin] RBAC: Authorizing `identity:get_group(group_id=62ddfa5186074f6eb9514f62cffa48da)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:31.766148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0be328-6afd-4988-be1d-4fcb80ab2121 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:31.768241 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 195/390] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jun 6 10:24:31 2022] GET /identity/v3/groups/62ddfa5186074f6eb9514f62cffa48da => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:31.788014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0be328-6afd-4988-be1d-4fcb80ab2121 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:31.945167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3178e-12aa-40bc-8d55-8522f8d071b2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:31.988134 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:31.991253 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0be328-6afd-4988-be1d-4fcb80ab2121 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:32.019991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc570075-717f-44d3-bc2e-26fc5792f078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:32.020812 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc570075-717f-44d3-bc2e-26fc5792f078 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:32.021098 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc570075-717f-44d3-bc2e-26fc5792f078 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:32.021376 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc570075-717f-44d3-bc2e-26fc5792f078 None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:32.024136 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc570075-717f-44d3-bc2e-26fc5792f078 None admin] RBAC: Authorizing `identity:get_project(project_id=0efa83f60031447ab87be4f6f1af6165)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:32.025604 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc570075-717f-44d3-bc2e-26fc5792f078 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:32.027950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 196/391] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:31 2022] GET /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:32.032782 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc570075-717f-44d3-bc2e-26fc5792f078 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:32.055494 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ac645-488e-4b55-9e22-102f1c368f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:32.056390 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ac645-488e-4b55-9e22-102f1c368f96 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:32.056677 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ac645-488e-4b55-9e22-102f1c368f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:32.056957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ac645-488e-4b55-9e22-102f1c368f96 None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:32.059558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ac645-488e-4b55-9e22-102f1c368f96 None admin] RBAC: Authorizing `identity:get_project(project_id=0efa83f60031447ab87be4f6f1af6165)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:32.061034 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ac645-488e-4b55-9e22-102f1c368f96 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:32.062752 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 196/392] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:31 2022] GET /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165 => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:32.080254 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:32.082165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ac645-488e-4b55-9e22-102f1c368f96 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:32.252056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc570075-717f-44d3-bc2e-26fc5792f078 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:32.280870 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ac645-488e-4b55-9e22-102f1c368f96 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:32.309165 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd393e0-06ee-49de-8898-9170e35b1b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:32.310056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd393e0-06ee-49de-8898-9170e35b1b8f None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:32.310343 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd393e0-06ee-49de-8898-9170e35b1b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:32.310617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd393e0-06ee-49de-8898-9170e35b1b8f None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165/groups/62ddfa5186074f6eb9514f62cffa48da/roles/f214f8dbc3354cfc81fa2c2f116980e0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:32.313098 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd393e0-06ee-49de-8898-9170e35b1b8f None admin] RBAC: Authorizing `identity:create_grant(project_id=0efa83f60031447ab87be4f6f1af6165, group_id=62ddfa5186074f6eb9514f62cffa48da, role_id=f214f8dbc3354cfc81fa2c2f116980e0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:32.314716 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd393e0-06ee-49de-8898-9170e35b1b8f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:32.337189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce738cb-2844-4b1c-b3d4-5e20b5e4768a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:32.338329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce738cb-2844-4b1c-b3d4-5e20b5e4768a None admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:32.338628 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce738cb-2844-4b1c-b3d4-5e20b5e4768a None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:32.338901 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce738cb-2844-4b1c-b3d4-5e20b5e4768a None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165/groups/62ddfa5186074f6eb9514f62cffa48da/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:32.341287 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce738cb-2844-4b1c-b3d4-5e20b5e4768a None admin] RBAC: Authorizing `identity:create_grant(project_id=0efa83f60031447ab87be4f6f1af6165, group_id=62ddfa5186074f6eb9514f62cffa48da, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:32.342960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce738cb-2844-4b1c-b3d4-5e20b5e4768a None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:32.354330 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 197/393] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Jun 6 10:24:32 2022] PUT /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165/groups/62ddfa5186074f6eb9514f62cffa48da/roles/f214f8dbc3354cfc81fa2c2f116980e0 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:24:32.360808 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 197/394] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Jun 6 10:24:32 2022] PUT /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165/groups/62ddfa5186074f6eb9514f62cffa48da/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:24:32.471226 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a4016-c27e-4d7c-99db-e50ddd4a6602 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:32.471508 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a4016-c27e-4d7c-99db-e50ddd4a6602 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:32.471810 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a4016-c27e-4d7c-99db-e50ddd4a6602 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:32.474262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 198/395] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:32.480420 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004275c3-aaf7-4db3-bdfa-c02dc25725ee None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:32.480705 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004275c3-aaf7-4db3-bdfa-c02dc25725ee None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:32.480988 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004275c3-aaf7-4db3-bdfa-c02dc25725ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:32.505257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-004275c3-aaf7-4db3-bdfa-c02dc25725ee None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:32.506503 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-004275c3-aaf7-4db3-bdfa-c02dc25725ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:32.539878 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:32.606824 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 198/396] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:32.630956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a4016-c27e-4d7c-99db-e50ddd4a6602 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:32.732559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:32.759770 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a4016-c27e-4d7c-99db-e50ddd4a6602 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:32.797042 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb46519-d28f-45e1-a9d6-82256e712805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:32.797829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb46519-d28f-45e1-a9d6-82256e712805 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:32.798112 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb46519-d28f-45e1-a9d6-82256e712805 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:32.798387 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb46519-d28f-45e1-a9d6-82256e712805 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:32.823374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb46519-d28f-45e1-a9d6-82256e712805 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:32.831348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-9bb46519-d28f-45e1-a9d6-82256e712805 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:32.831348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:32.831348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:32.831348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:32.831348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:32.831348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:32.831348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:32.831348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:32.831348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:32.831348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:32.831348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:32.831348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:32.831348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:32.831348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:32.831348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:32.831348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:32.832303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:32.832303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:32.832303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:32.832303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:32.832303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:32.832303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:32.832303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:32.832303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:32.832303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:32.832303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:32.832303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:32.832303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:32.832303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:32.832303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:32.832303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:32.832303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:32.833385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:32.833385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:32.833385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:32.833385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:32.833385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:32.833385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:32.833385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:32.833385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:32.833385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:32.833385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:32.833385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:32.833385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:32.837763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 199/397] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:32.840908 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004275c3-aaf7-4db3-bdfa-c02dc25725ee None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:32.991187 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004275c3-aaf7-4db3-bdfa-c02dc25725ee None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:33.001736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542f683-c79c-4db4-8e4d-9ec3fd6b8277 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:33.002021 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542f683-c79c-4db4-8e4d-9ec3fd6b8277 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:33.002300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542f683-c79c-4db4-8e4d-9ec3fd6b8277 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:33.003820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 200/398] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:24:33.008725 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5c757-f523-4f64-94e0-2156d42d4d2d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:33.009009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5c757-f523-4f64-94e0-2156d42d4d2d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:33.009283 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5c757-f523-4f64-94e0-2156d42d4d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:33.040785 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-d2d5c757-f523-4f64-94e0-2156d42d4d2d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:33.042904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d2d5c757-f523-4f64-94e0-2156d42d4d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:33.056140 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8174d-2868-4fc2-b3b5-185794c2d595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:33.056747 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8174d-2868-4fc2-b3b5-185794c2d595 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:33.057031 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8174d-2868-4fc2-b3b5-185794c2d595 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:33.057310 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8174d-2868-4fc2-b3b5-185794c2d595 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:33.058113 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8174d-2868-4fc2-b3b5-185794c2d595 None admin] RBAC: Adding query filter params (name=member) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:33.058470 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8174d-2868-4fc2-b3b5-185794c2d595 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:33.060156 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8174d-2868-4fc2-b3b5-185794c2d595 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:33.073328 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 199/399] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:24:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:33.081096 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8174d-2868-4fc2-b3b5-185794c2d595 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:33.120226 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:33.179925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 201/400] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 10:24:33.181530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913176b-f486-4af4-9e30-5f38f8eda134 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:33.182083 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913176b-f486-4af4-9e30-5f38f8eda134 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:33.182557 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913176b-f486-4af4-9e30-5f38f8eda134 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:33.184070 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 202/401] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:33.197070 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8771b77-41d7-47ce-b231-8d6062220e6f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:33.197643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8771b77-41d7-47ce-b231-8d6062220e6f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:33.198189 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8771b77-41d7-47ce-b231-8d6062220e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:33.198514 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8174d-2868-4fc2-b3b5-185794c2d595 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:33.226006 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c8771b77-41d7-47ce-b231-8d6062220e6f None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:33.227565 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c8771b77-41d7-47ce-b231-8d6062220e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:33.238184 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e09b31-dff4-4642-a939-6b5b59cdb11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:33.239536 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e09b31-dff4-4642-a939-6b5b59cdb11c None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:33.240006 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e09b31-dff4-4642-a939-6b5b59cdb11c None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:33.240360 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e09b31-dff4-4642-a939-6b5b59cdb11c None admin] PATH_INFO: `/v3/groups/62ddfa5186074f6eb9514f62cffa48da` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:33.242251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e09b31-dff4-4642-a939-6b5b59cdb11c None admin] RBAC: Authorizing `identity:get_group(group_id=62ddfa5186074f6eb9514f62cffa48da)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:33.243975 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e09b31-dff4-4642-a939-6b5b59cdb11c None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:33.245667 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 200/402] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jun 6 10:24:33 2022] GET /identity/v3/groups/62ddfa5186074f6eb9514f62cffa48da => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:33.246436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e09b31-dff4-4642-a939-6b5b59cdb11c None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:33.310241 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:33.316602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 203/403] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:33.318284 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8771b77-41d7-47ce-b231-8d6062220e6f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:33.388653 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e09b31-dff4-4642-a939-6b5b59cdb11c None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:33.430011 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b39409-e53d-4752-a476-fa6098d409c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:33.430840 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b39409-e53d-4752-a476-fa6098d409c3 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:33.431124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b39409-e53d-4752-a476-fa6098d409c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:33.431400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b39409-e53d-4752-a476-fa6098d409c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:33.445206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b39409-e53d-4752-a476-fa6098d409c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:33.453351 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:33.456604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-e9b39409-e53d-4752-a476-fa6098d409c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:33.456604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:33.456604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:33.456604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:33.456604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:33.456604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:33.456604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:33.456604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:33.456604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:33.456604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:33.456604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:33.456604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:33.456604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:33.456604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:33.456604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:33.456604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:33.458200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:33.458200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:33.458200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:33.458200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:33.458200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:33.458200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:33.458200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:33.458200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:33.458200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:33.458200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:33.458200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:33.458200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:33.458200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:33.458200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:33.458200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:33.458200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:33.459206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:33.459206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:33.459206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:33.459206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:33.459206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:33.459206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:33.459206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:33.459206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:33.459206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:33.459206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:33.459206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:33.459206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:33.463926 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 201/404] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:24:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:33.465640 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b39409-e53d-4752-a476-fa6098d409c3 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:33.468932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8771b77-41d7-47ce-b231-8d6062220e6f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:33.517006 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd2025-12ba-4ee2-8db4-d5f45d9883cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:33.518689 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd2025-12ba-4ee2-8db4-d5f45d9883cf None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:33.519195 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd2025-12ba-4ee2-8db4-d5f45d9883cf None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:33.520674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd2025-12ba-4ee2-8db4-d5f45d9883cf None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:33.522366 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd2025-12ba-4ee2-8db4-d5f45d9883cf None admin] RBAC: Authorizing `identity:get_project(project_id=d65d04e96c4b4b87be63be85bb9731a0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:33.524799 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd2025-12ba-4ee2-8db4-d5f45d9883cf None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:33.526575 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 204/405] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:33 2022] GET /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:33.527347 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd2025-12ba-4ee2-8db4-d5f45d9883cf None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:33.592309 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b39409-e53d-4752-a476-fa6098d409c3 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:33.605955 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:33.637022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae68914-5f0f-490f-bf30-9c1340571c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:33.638188 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae68914-5f0f-490f-bf30-9c1340571c59 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:33.638671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae68914-5f0f-490f-bf30-9c1340571c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:33.639152 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae68914-5f0f-490f-bf30-9c1340571c59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:33.649833 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd2025-12ba-4ee2-8db4-d5f45d9883cf None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:33.651328 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae68914-5f0f-490f-bf30-9c1340571c59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:33.659970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-9ae68914-5f0f-490f-bf30-9c1340571c59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:24:33.659970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:33.659970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:33.659970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:33.659970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:33.659970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:33.659970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:33.659970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:33.659970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:33.659970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:33.659970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:33.659970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:33.659970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:33.659970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:33.659970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:33.659970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:33.660886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:33.660886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:33.660886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:33.660886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:33.660886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:33.660886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:33.660886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:33.660886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:33.660886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:33.660886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:33.660886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:33.660886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:33.660886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:33.660886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:33.660886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:33.660886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:33.661702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:33.661702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:33.661702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:33.661702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:33.661702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:33.661702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:33.661702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:33.661702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:33.661702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:33.661702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:33.661702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:24:33.661702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:33.668977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 202/406] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:24:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:33.670053 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae68914-5f0f-490f-bf30-9c1340571c59 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:33.681446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:33.686844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c3d33-69f9-451c-a68b-12c8d77507ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:33.687760 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c3d33-69f9-451c-a68b-12c8d77507ca None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:33.688049 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c3d33-69f9-451c-a68b-12c8d77507ca None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:33.688354 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c3d33-69f9-451c-a68b-12c8d77507ca None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:33.689266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c3d33-69f9-451c-a68b-12c8d77507ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:33.689676 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c3d33-69f9-451c-a68b-12c8d77507ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:33.690967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c3d33-69f9-451c-a68b-12c8d77507ca None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:33.705154 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 205/407] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:24:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:33.705933 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c3d33-69f9-451c-a68b-12c8d77507ca None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:33.788702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae68914-5f0f-490f-bf30-9c1340571c59 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:33.813239 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c3d33-69f9-451c-a68b-12c8d77507ca None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:33.829217 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23663e2c-3aa5-45eb-8158-b5f5b972b45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:33.830541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23663e2c-3aa5-45eb-8158-b5f5b972b45c None admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:33.831034 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23663e2c-3aa5-45eb-8158-b5f5b972b45c None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:33.831535 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23663e2c-3aa5-45eb-8158-b5f5b972b45c None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0/groups/62ddfa5186074f6eb9514f62cffa48da/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:33.834247 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23663e2c-3aa5-45eb-8158-b5f5b972b45c None admin] RBAC: Authorizing `identity:create_grant(project_id=d65d04e96c4b4b87be63be85bb9731a0, group_id=62ddfa5186074f6eb9514f62cffa48da, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:33.836004 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23663e2c-3aa5-45eb-8158-b5f5b972b45c None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:33.847474 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:33.855532 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0477a3e-c710-4a24-a4de-195b1f4339c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:33.856255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0477a3e-c710-4a24-a4de-195b1f4339c7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:33.856541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0477a3e-c710-4a24-a4de-195b1f4339c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:33.856818 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0477a3e-c710-4a24-a4de-195b1f4339c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:33.857481 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 203/408] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Jun 6 10:24:33 2022] PUT /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0/groups/62ddfa5186074f6eb9514f62cffa48da/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:24:33.857926 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0477a3e-c710-4a24-a4de-195b1f4339c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:33.858031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0477a3e-c710-4a24-a4de-195b1f4339c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:33.859119 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23663e2c-3aa5-45eb-8158-b5f5b972b45c None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:33.859485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0477a3e-c710-4a24-a4de-195b1f4339c7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:33.873772 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 206/409] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:33.881145 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0477a3e-c710-4a24-a4de-195b1f4339c7 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:33.962547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:34.011810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23663e2c-3aa5-45eb-8158-b5f5b972b45c None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:34.055548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c88613-e283-4b41-8cf6-d2d3832ccefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:34.056837 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c88613-e283-4b41-8cf6-d2d3832ccefe None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:34.057319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c88613-e283-4b41-8cf6-d2d3832ccefe None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:34.057791 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c88613-e283-4b41-8cf6-d2d3832ccefe None admin] PATH_INFO: `/v3/groups/62ddfa5186074f6eb9514f62cffa48da` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:34.060006 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c88613-e283-4b41-8cf6-d2d3832ccefe None admin] RBAC: Authorizing `identity:get_group(group_id=62ddfa5186074f6eb9514f62cffa48da)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:34.061720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c88613-e283-4b41-8cf6-d2d3832ccefe None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:34.063605 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 204/410] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jun 6 10:24:33 2022] GET /identity/v3/groups/62ddfa5186074f6eb9514f62cffa48da => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:34.073607 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c88613-e283-4b41-8cf6-d2d3832ccefe None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:34.120808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0477a3e-c710-4a24-a4de-195b1f4339c7 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:34.189914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555f9ea-b837-45a0-9359-df4c33583fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:34.191660 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555f9ea-b837-45a0-9359-df4c33583fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:34.193551 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555f9ea-b837-45a0-9359-df4c33583fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:34.193834 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555f9ea-b837-45a0-9359-df4c33583fc9 None admin] PATH_INFO: `/v3/groups/83a537bba06a44fbb6d102d17532495d` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:34.195204 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1555f9ea-b837-45a0-9359-df4c33583fc9 None admin] RBAC: Authorizing `identity:get_group(group_id=83a537bba06a44fbb6d102d17532495d)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:34.196669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1555f9ea-b837-45a0-9359-df4c33583fc9 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:34.199142 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 207/411] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jun 6 10:24:33 2022] GET /identity/v3/groups/83a537bba06a44fbb6d102d17532495d => generated 229 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:34.205051 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c88613-e283-4b41-8cf6-d2d3832ccefe None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:34.207151 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555f9ea-b837-45a0-9359-df4c33583fc9 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:34.291268 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:34.292546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb4663-3651-4f44-8f06-7ba7dace65cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:34.293436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb4663-3651-4f44-8f06-7ba7dace65cf None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:34.293779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb4663-3651-4f44-8f06-7ba7dace65cf None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:34.294143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb4663-3651-4f44-8f06-7ba7dace65cf None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:34.295405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb4663-3651-4f44-8f06-7ba7dace65cf None admin] RBAC: Authorizing `identity:get_project(project_id=d65d04e96c4b4b87be63be85bb9731a0)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:34.296890 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb4663-3651-4f44-8f06-7ba7dace65cf None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:34.298955 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 205/412] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:34 2022] GET /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:34.304587 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb4663-3651-4f44-8f06-7ba7dace65cf None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:34.309494 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:34.336250 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555f9ea-b837-45a0-9359-df4c33583fc9 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:34.373346 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609eaf90-45f5-4bd1-806e-f8c5a3cc1b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:34.374129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609eaf90-45f5-4bd1-806e-f8c5a3cc1b6f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:34.374408 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609eaf90-45f5-4bd1-806e-f8c5a3cc1b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:34.374684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609eaf90-45f5-4bd1-806e-f8c5a3cc1b6f None admin] PATH_INFO: `/v3/projects/863195bccbf14e04a9a61a90f52ba9f1` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:34.376007 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609eaf90-45f5-4bd1-806e-f8c5a3cc1b6f None admin] RBAC: Authorizing `identity:get_project(project_id=863195bccbf14e04a9a61a90f52ba9f1)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:34.377461 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609eaf90-45f5-4bd1-806e-f8c5a3cc1b6f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:34.379270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 208/413] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:34 2022] GET /identity/v3/projects/863195bccbf14e04a9a61a90f52ba9f1 => generated 344 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:34.390054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609eaf90-45f5-4bd1-806e-f8c5a3cc1b6f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:34.505851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb4663-3651-4f44-8f06-7ba7dace65cf None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:34.574208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609eaf90-45f5-4bd1-806e-f8c5a3cc1b6f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:34.585051 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e95cfd-9aad-46d4-902b-8909d4b27455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:34.585943 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e95cfd-9aad-46d4-902b-8909d4b27455 None admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:34.586226 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e95cfd-9aad-46d4-902b-8909d4b27455 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:34.586500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e95cfd-9aad-46d4-902b-8909d4b27455 None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0/groups/62ddfa5186074f6eb9514f62cffa48da/roles/f214f8dbc3354cfc81fa2c2f116980e0` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:34.596562 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e95cfd-9aad-46d4-902b-8909d4b27455 None admin] RBAC: Authorizing `identity:create_grant(project_id=d65d04e96c4b4b87be63be85bb9731a0, group_id=62ddfa5186074f6eb9514f62cffa48da, role_id=f214f8dbc3354cfc81fa2c2f116980e0)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:34.598010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e95cfd-9aad-46d4-902b-8909d4b27455 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:34.606748 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:34.612912 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd65e23-2a15-4164-b2e8-5c293d815001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:34.613974 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd65e23-2a15-4164-b2e8-5c293d815001 None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:34.614447 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd65e23-2a15-4164-b2e8-5c293d815001 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:34.615010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd65e23-2a15-4164-b2e8-5c293d815001 None admin] PATH_INFO: `/v3/projects/863195bccbf14e04a9a61a90f52ba9f1/groups/83a537bba06a44fbb6d102d17532495d/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:34.617597 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd65e23-2a15-4164-b2e8-5c293d815001 None admin] RBAC: Authorizing `identity:create_grant(project_id=863195bccbf14e04a9a61a90f52ba9f1, group_id=83a537bba06a44fbb6d102d17532495d, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:34.619365 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd65e23-2a15-4164-b2e8-5c293d815001 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:34.635735 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 206/415] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Jun 6 10:24:34 2022] PUT /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0/groups/62ddfa5186074f6eb9514f62cffa48da/roles/f214f8dbc3354cfc81fa2c2f116980e0 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:24:34.635931 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 209/415] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Jun 6 10:24:34 2022] PUT /identity/v3/projects/863195bccbf14e04a9a61a90f52ba9f1/groups/83a537bba06a44fbb6d102d17532495d/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:24:35.038498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95eb3f0-a217-4ffe-afb5-d07acf9fc4f5 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:35.039486 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95eb3f0-a217-4ffe-afb5-d07acf9fc4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:35.040027 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95eb3f0-a217-4ffe-afb5-d07acf9fc4f5 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:35.041846 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 207/416] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:35.051887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c7970-152c-4a41-8385-d11efa52d068 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:35.052356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c7970-152c-4a41-8385-d11efa52d068 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:35.052461 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c7970-152c-4a41-8385-d11efa52d068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:35.072302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-236c7970-152c-4a41-8385-d11efa52d068 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:35.073487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-236c7970-152c-4a41-8385-d11efa52d068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:35.181499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 210/417] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:35.216517 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95eb3f0-a217-4ffe-afb5-d07acf9fc4f5 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:35.265650 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038951e-e89f-4a65-b56d-d70a57ef1562 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:35.265994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038951e-e89f-4a65-b56d-d70a57ef1562 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:35.266207 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038951e-e89f-4a65-b56d-d70a57ef1562 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:35.267815 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 211/418] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:24:35.286684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44c76f-87cb-4774-a8d2-87b9373842a4 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:35.287326 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44c76f-87cb-4774-a8d2-87b9373842a4 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:35.287827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44c76f-87cb-4774-a8d2-87b9373842a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:35.305723 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:35.322082 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-dc44c76f-87cb-4774-a8d2-87b9373842a4 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:35.323657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-dc44c76f-87cb-4774-a8d2-87b9373842a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:35.354266 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95eb3f0-a217-4ffe-afb5-d07acf9fc4f5 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:35.403375 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd483743-f125-43d8-8fd3-b08f3a990c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:35.404231 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd483743-f125-43d8-8fd3-b08f3a990c7b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:35.404513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd483743-f125-43d8-8fd3-b08f3a990c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:35.405870 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd483743-f125-43d8-8fd3-b08f3a990c7b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:35.440414 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd483743-f125-43d8-8fd3-b08f3a990c7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:35.449257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-bd483743-f125-43d8-8fd3-b08f3a990c7b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:35.449257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:35.449257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:35.449257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:35.449257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:35.449257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:35.449257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:35.449257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:35.449257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:35.449257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:35.449257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:35.449257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:35.449257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:35.449257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:35.449257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:35.449257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:35.450148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:35.450148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:35.450148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:35.450148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:35.450148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:35.450148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:35.450148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:35.450148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:35.450148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:35.450148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:35.450148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:35.450148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:35.450148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:35.450148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:35.450148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:35.450148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:35.450947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:35.450947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:35.450947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:35.450947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:35.450947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:35.450947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:35.450947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:35.450947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:35.450947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:35.450947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:35.450947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:35.450947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:35.450947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 208/419] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:24:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:35.456503 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd483743-f125-43d8-8fd3-b08f3a990c7b None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:35.461445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 212/420] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:35.485681 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44c76f-87cb-4774-a8d2-87b9373842a4 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:35.544400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:35.611148 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44c76f-87cb-4774-a8d2-87b9373842a4 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:35.617187 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd483743-f125-43d8-8fd3-b08f3a990c7b None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:35.660162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:35.675082 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be966589-8f96-4960-987c-e35bf3094e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:35.676070 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be966589-8f96-4960-987c-e35bf3094e39 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:35.676924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be966589-8f96-4960-987c-e35bf3094e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:35.677406 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be966589-8f96-4960-987c-e35bf3094e39 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:35.678434 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be966589-8f96-4960-987c-e35bf3094e39 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:35.678951 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be966589-8f96-4960-987c-e35bf3094e39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:35.680489 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d82d20-08eb-4036-b4b3-d9b5da9639a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:35.680772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be966589-8f96-4960-987c-e35bf3094e39 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:35.681298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d82d20-08eb-4036-b4b3-d9b5da9639a7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:35.681583 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d82d20-08eb-4036-b4b3-d9b5da9639a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:35.681859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d82d20-08eb-4036-b4b3-d9b5da9639a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:35.690998 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 209/421] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:24:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:35.700733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be966589-8f96-4960-987c-e35bf3094e39 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:35.709388 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d82d20-08eb-4036-b4b3-d9b5da9639a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:35.722244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-c6d82d20-08eb-4036-b4b3-d9b5da9639a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:35.722244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:35.722244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:35.722244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:35.722244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:35.722244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:35.722244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:35.722244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:35.722244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:35.722244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:35.722244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:35.722244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:35.722244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:35.722244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:35.722244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:35.722244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:35.723411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:35.723411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:35.723411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:35.723411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:35.723411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:35.723411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:35.723411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:35.723411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:35.723411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:35.723411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:35.723411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:35.723411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:35.723411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:35.723411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:35.723411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:35.723411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:35.724321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:35.724321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:35.724321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:35.724321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:35.724321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:35.724321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:35.724321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:35.724321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:35.724321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:35.724321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:35.724321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:35.724321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:35.726573 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 213/422] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:24:35.736126 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d82d20-08eb-4036-b4b3-d9b5da9639a7 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:35.861398 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be966589-8f96-4960-987c-e35bf3094e39 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:35.930710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c65750a-900b-421b-83c7-f4f28f9842d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:35.931372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65750a-900b-421b-83c7-f4f28f9842d0 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:35.931654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65750a-900b-421b-83c7-f4f28f9842d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:35.931965 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65750a-900b-421b-83c7-f4f28f9842d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:35.941563 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d82d20-08eb-4036-b4b3-d9b5da9639a7 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:35.961927 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:36.008216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:36.024209 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65750a-900b-421b-83c7-f4f28f9842d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:36.025774 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65750a-900b-421b-83c7-f4f28f9842d0 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:36.040507 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c996d-8d28-46c9-9170-186b311f1d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:36.041193 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c996d-8d28-46c9-9170-186b311f1d67 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:36.041474 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c996d-8d28-46c9-9170-186b311f1d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:36.041745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c996d-8d28-46c9-9170-186b311f1d67 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:36.042494 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c996d-8d28-46c9-9170-186b311f1d67 None admin] RBAC: Adding query filter params (name=member) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:36.042797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c996d-8d28-46c9-9170-186b311f1d67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:36.044220 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c996d-8d28-46c9-9170-186b311f1d67 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:36.059376 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 214/423] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:24:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:36.065775 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c996d-8d28-46c9-9170-186b311f1d67 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:36.130413 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 210/424] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:36.135742 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c65750a-900b-421b-83c7-f4f28f9842d0 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:36.234939 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:36.257306 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c65750a-900b-421b-83c7-f4f28f9842d0 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:36.298460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fbd9fe-7055-4b77-8dd4-8291f8f1360b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:36.299347 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbd9fe-7055-4b77-8dd4-8291f8f1360b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:36.299630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbd9fe-7055-4b77-8dd4-8291f8f1360b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:36.299987 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbd9fe-7055-4b77-8dd4-8291f8f1360b None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:36.301412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fbd9fe-7055-4b77-8dd4-8291f8f1360b None admin] RBAC: Authorizing `identity:get_project(project_id=0efa83f60031447ab87be4f6f1af6165)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:36.302891 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fbd9fe-7055-4b77-8dd4-8291f8f1360b None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:36.304980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 211/425] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:36 2022] GET /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:36.313164 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fbd9fe-7055-4b77-8dd4-8291f8f1360b None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:36.401726 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c996d-8d28-46c9-9170-186b311f1d67 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:36.423916 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fbd9fe-7055-4b77-8dd4-8291f8f1360b None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:36.461493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266e5bd-b92b-4684-b03f-d12e8c690202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:36.462352 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266e5bd-b92b-4684-b03f-d12e8c690202 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:36.462634 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266e5bd-b92b-4684-b03f-d12e8c690202 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:36.462904 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266e5bd-b92b-4684-b03f-d12e8c690202 None admin] PATH_INFO: `/v3/groups/62ddfa5186074f6eb9514f62cffa48da` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:36.464111 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1266e5bd-b92b-4684-b03f-d12e8c690202 None admin] RBAC: Authorizing `identity:get_group(group_id=62ddfa5186074f6eb9514f62cffa48da)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:36.465545 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1266e5bd-b92b-4684-b03f-d12e8c690202 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:36.467946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 212/426] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jun 6 10:24:36 2022] GET /identity/v3/groups/62ddfa5186074f6eb9514f62cffa48da => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:36.473992 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266e5bd-b92b-4684-b03f-d12e8c690202 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:36.496698 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:36.530920 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0143d-24f4-489b-869e-7115bdcbdaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:36.531578 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0143d-24f4-489b-869e-7115bdcbdaef None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:36.531900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0143d-24f4-489b-869e-7115bdcbdaef None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:36.532179 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0143d-24f4-489b-869e-7115bdcbdaef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:36.605480 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:36.634342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266e5bd-b92b-4684-b03f-d12e8c690202 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:36.646638 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0143d-24f4-489b-869e-7115bdcbdaef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:36.648501 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0143d-24f4-489b-869e-7115bdcbdaef None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:36.678206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a152164-d65a-4417-afaf-5d95e8429276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:36.679136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a152164-d65a-4417-afaf-5d95e8429276 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:36.679423 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a152164-d65a-4417-afaf-5d95e8429276 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:36.679697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a152164-d65a-4417-afaf-5d95e8429276 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:36.680834 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a152164-d65a-4417-afaf-5d95e8429276 None admin] RBAC: Adding query filter params (group.id=62ddfa5186074f6eb9514f62cffa48da, role.id=f214f8dbc3354cfc81fa2c2f116980e0, scope.project.id=0efa83f60031447ab87be4f6f1af6165) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:36.681565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a152164-d65a-4417-afaf-5d95e8429276 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:36.683260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a152164-d65a-4417-afaf-5d95e8429276 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:36.696005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 213/427] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jun 6 10:24:36 2022] GET /identity/v3/role_assignments?group.id=62ddfa5186074f6eb9514f62cffa48da&scope.project.id=0efa83f60031447ab87be4f6f1af6165&role.id=f214f8dbc3354cfc81fa2c2f116980e0 => generated 611 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:36.697615 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738a525-7f51-435c-9979-14f853db1ed7 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:36.698019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738a525-7f51-435c-9979-14f853db1ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:36.698393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738a525-7f51-435c-9979-14f853db1ed7 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:36.700352 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 214/428] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:36.713671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e9f96-6801-42a6-9699-798479d0b4e2 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:36.714006 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e9f96-6801-42a6-9699-798479d0b4e2 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:36.714222 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e9f96-6801-42a6-9699-798479d0b4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:36.743729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-ff5e9f96-6801-42a6-9699-798479d0b4e2 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:36.745367 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ff5e9f96-6801-42a6-9699-798479d0b4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:36.836499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 215/429] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 771 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 06 10:24:36.842160 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0143d-24f4-489b-869e-7115bdcbdaef None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:36.881427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 215/430] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 06 10:24:36.924779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e9f96-6801-42a6-9699-798479d0b4e2 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:36.948633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:36.952245 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:37.015483 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0143d-24f4-489b-869e-7115bdcbdaef None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:37.066921 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e981a8-fe37-4c05-9d2f-56123948386c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:37.068175 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e981a8-fe37-4c05-9d2f-56123948386c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:37.068462 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e981a8-fe37-4c05-9d2f-56123948386c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:37.068990 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e981a8-fe37-4c05-9d2f-56123948386c None admin] PATH_INFO: `/v3/projects/0efa83f60031447ab87be4f6f1af6165` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:37.070692 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e981a8-fe37-4c05-9d2f-56123948386c None admin] RBAC: Authorizing `identity:get_project(project_id=0efa83f60031447ab87be4f6f1af6165)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:37.073608 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e981a8-fe37-4c05-9d2f-56123948386c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:37.076330 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 216/431] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:36 2022] GET /identity/v3/projects/0efa83f60031447ab87be4f6f1af6165 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:37.082102 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e9f96-6801-42a6-9699-798479d0b4e2 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:37.084927 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e981a8-fe37-4c05-9d2f-56123948386c None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:37.131554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aee465-69bc-4a8a-8238-57d73c85827d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:37.132402 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aee465-69bc-4a8a-8238-57d73c85827d None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:37.132682 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aee465-69bc-4a8a-8238-57d73c85827d None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:37.132955 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aee465-69bc-4a8a-8238-57d73c85827d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:37.147092 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aee465-69bc-4a8a-8238-57d73c85827d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:37.154899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-55aee465-69bc-4a8a-8238-57d73c85827d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:37.154899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:37.154899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:37.154899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:37.154899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:37.154899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:37.154899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:37.154899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:37.154899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:37.154899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:37.154899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:37.154899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:37.154899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:37.154899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:37.154899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:37.154899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:37.155768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:37.155768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:37.155768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:37.155768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:37.155768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:37.155768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:37.155768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:37.155768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:37.155768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:37.155768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:37.155768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:37.155768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:37.155768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:37.155768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:37.155768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:37.155768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:37.157003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:37.157003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:37.157003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:37.157003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:37.157003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:37.157003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:37.157003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:37.157003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:37.157003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:37.157003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:37.157003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 10:24:37.157003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:37.157003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 216/432] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:24:37.164453 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aee465-69bc-4a8a-8238-57d73c85827d None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:37.218409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e981a8-fe37-4c05-9d2f-56123948386c None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:37.255180 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9cf47-5eac-4455-99ae-2c6b57bce5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:37.256075 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9cf47-5eac-4455-99ae-2c6b57bce5e1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:37.256357 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9cf47-5eac-4455-99ae-2c6b57bce5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:37.256626 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9cf47-5eac-4455-99ae-2c6b57bce5e1 None admin] PATH_INFO: `/v3/groups/62ddfa5186074f6eb9514f62cffa48da` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:37.257823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9cf47-5eac-4455-99ae-2c6b57bce5e1 None admin] RBAC: Authorizing `identity:get_group(group_id=62ddfa5186074f6eb9514f62cffa48da)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:37.259292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9cf47-5eac-4455-99ae-2c6b57bce5e1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:37.261254 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 217/433] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jun 6 10:24:37 2022] GET /identity/v3/groups/62ddfa5186074f6eb9514f62cffa48da => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:37.266829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9cf47-5eac-4455-99ae-2c6b57bce5e1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:37.271270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:37.276048 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:37.295725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aee465-69bc-4a8a-8238-57d73c85827d None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:37.331076 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80483eb5-5036-42a1-a310-cbe900979078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:37.331735 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80483eb5-5036-42a1-a310-cbe900979078 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:37.332047 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80483eb5-5036-42a1-a310-cbe900979078 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:37.332357 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80483eb5-5036-42a1-a310-cbe900979078 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:37.333248 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80483eb5-5036-42a1-a310-cbe900979078 None admin] RBAC: Adding query filter params (name=member) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:37.333740 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80483eb5-5036-42a1-a310-cbe900979078 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:37.335120 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80483eb5-5036-42a1-a310-cbe900979078 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:37.344231 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 217/434] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:24:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:37.345769 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db5c8a-45ec-4d3e-8f94-6f4a2e8eaf34 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:37.346291 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db5c8a-45ec-4d3e-8f94-6f4a2e8eaf34 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:37.347192 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db5c8a-45ec-4d3e-8f94-6f4a2e8eaf34 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:37.349561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 218/435] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:37.350489 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db5c8a-45ec-4d3e-8f94-6f4a2e8eaf34 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:37.375376 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9cf47-5eac-4455-99ae-2c6b57bce5e1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:37.418620 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caed242-39ab-453c-a2d3-624f3ad4ca09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:37.419272 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caed242-39ab-453c-a2d3-624f3ad4ca09 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:37.419554 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caed242-39ab-453c-a2d3-624f3ad4ca09 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:37.419860 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caed242-39ab-453c-a2d3-624f3ad4ca09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:37.420692 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caed242-39ab-453c-a2d3-624f3ad4ca09 None admin] RBAC: Adding query filter params (group.id=62ddfa5186074f6eb9514f62cffa48da, role.id=6bcb06bf4aa4442ebcfebeadf1af41de, scope.project.id=0efa83f60031447ab87be4f6f1af6165) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:37.421003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caed242-39ab-453c-a2d3-624f3ad4ca09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:37.422441 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caed242-39ab-453c-a2d3-624f3ad4ca09 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:37.428627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 218/436] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jun 6 10:24:37 2022] GET /identity/v3/role_assignments?group.id=62ddfa5186074f6eb9514f62cffa48da&scope.project.id=0efa83f60031447ab87be4f6f1af6165&role.id=6bcb06bf4aa4442ebcfebeadf1af41de => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:37.434491 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5843eb-bc62-414c-99db-92dddd7d1a64 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:37.435101 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5843eb-bc62-414c-99db-92dddd7d1a64 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:37.435600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5843eb-bc62-414c-99db-92dddd7d1a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:37.456023 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2b5843eb-bc62-414c-99db-92dddd7d1a64 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:37.457273 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2b5843eb-bc62-414c-99db-92dddd7d1a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:37.478440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db5c8a-45ec-4d3e-8f94-6f4a2e8eaf34 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:37.516169 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154bd03-ab01-4c69-bd75-50e062ea738d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:37.516822 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154bd03-ab01-4c69-bd75-50e062ea738d None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:37.517105 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154bd03-ab01-4c69-bd75-50e062ea738d None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:37.517380 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154bd03-ab01-4c69-bd75-50e062ea738d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:37.524020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:37.533944 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:37.542824 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 219/437] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:37.544221 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8f7a5-72a8-4ecf-93d6-213df490036c None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:37.544506 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8f7a5-72a8-4ecf-93d6-213df490036c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:37.544773 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8f7a5-72a8-4ecf-93d6-213df490036c None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:37.550298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 220/438] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:37.552496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed9ebe-77c0-4efc-8418-6618f71945d0 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:37.552787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed9ebe-77c0-4efc-8418-6618f71945d0 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:37.553069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed9ebe-77c0-4efc-8418-6618f71945d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:37.570048 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b5ed9ebe-77c0-4efc-8418-6618f71945d0 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:37.571279 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b5ed9ebe-77c0-4efc-8418-6618f71945d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:37.612550 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154bd03-ab01-4c69-bd75-50e062ea738d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:37.614090 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154bd03-ab01-4c69-bd75-50e062ea738d None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:37.650960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 221/439] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 10:24:37.651717 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed9ebe-77c0-4efc-8418-6618f71945d0 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:37.739216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 219/440] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:37.740370 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154bd03-ab01-4c69-bd75-50e062ea738d None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:37.768336 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed9ebe-77c0-4efc-8418-6618f71945d0 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:37.780896 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:37.812701 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:37.822317 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d7782-e572-4196-bd2d-8d7aaceb5006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:37.823126 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d7782-e572-4196-bd2d-8d7aaceb5006 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:37.823411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d7782-e572-4196-bd2d-8d7aaceb5006 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:37.823688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d7782-e572-4196-bd2d-8d7aaceb5006 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:37.838186 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d7782-e572-4196-bd2d-8d7aaceb5006 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:37.846031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-936d7782-e572-4196-bd2d-8d7aaceb5006 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:24:37.846031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:37.846031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:37.846031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:37.846031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:37.846031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:37.846031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:37.846031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:37.846031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:37.846031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:37.846031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:37.846031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:37.846031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:37.846031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:37.846031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:37.846031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:37.846932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:37.846932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:37.846932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:37.846932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:37.846932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:37.846932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:37.846932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:37.846932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:37.846932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:37.846932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:37.846932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:37.846932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:37.846932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:37.846932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:37.846932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:37.846932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:37.847721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:37.847721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:37.847721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:37.847721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:37.847721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:37.847721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:37.847721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:37.847721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:37.847721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:37.847721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:37.847721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:24:37.847721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:37.847721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 222/441] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:24:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:37.851222 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d7782-e572-4196-bd2d-8d7aaceb5006 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:37.855829 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154bd03-ab01-4c69-bd75-50e062ea738d None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:37.903748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1adff-35f7-4382-9907-9af3371df80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:37.903748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1adff-35f7-4382-9907-9af3371df80c None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:37.903748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1adff-35f7-4382-9907-9af3371df80c None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:37.903748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1adff-35f7-4382-9907-9af3371df80c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:37.918149 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1adff-35f7-4382-9907-9af3371df80c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:37.926671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-b3f1adff-35f7-4382-9907-9af3371df80c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:37.926671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:37.926671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:37.926671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:37.926671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:37.926671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:37.926671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:37.926671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:37.926671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:37.926671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:37.926671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:37.926671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:37.926671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:37.926671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:37.926671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:37.926671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:37.927602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:37.927602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:37.927602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:37.927602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:37.927602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:37.927602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:37.927602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:37.927602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:37.927602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:37.927602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:37.927602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:37.927602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:37.927602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:37.927602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:37.927602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:37.927602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:37.928438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:37.928438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:37.928438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:37.928438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:37.928438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:37.928438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:37.928438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:37.928438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:37.928438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:37.928438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:37.928438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:37.928438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:37.928438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 220/442] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:24:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:37.929108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1adff-35f7-4382-9907-9af3371df80c None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:37.983524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d7782-e572-4196-bd2d-8d7aaceb5006 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:38.022797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72f6f0-764f-4f96-a442-90dca25ffd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:38.023588 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72f6f0-764f-4f96-a442-90dca25ffd79 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:38.024144 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72f6f0-764f-4f96-a442-90dca25ffd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:38.024421 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72f6f0-764f-4f96-a442-90dca25ffd79 None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:38.025579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df72f6f0-764f-4f96-a442-90dca25ffd79 None admin] RBAC: Authorizing `identity:get_project(project_id=d65d04e96c4b4b87be63be85bb9731a0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:38.027021 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df72f6f0-764f-4f96-a442-90dca25ffd79 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:38.028730 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 223/443] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:37 2022] GET /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:38.029474 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72f6f0-764f-4f96-a442-90dca25ffd79 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:38.037948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:38.044367 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1adff-35f7-4382-9907-9af3371df80c None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:38.077374 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f4c06-8a2a-41d2-9891-9e9ae8331744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:38.078032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f4c06-8a2a-41d2-9891-9e9ae8331744 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:38.078315 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f4c06-8a2a-41d2-9891-9e9ae8331744 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:38.078592 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f4c06-8a2a-41d2-9891-9e9ae8331744 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:38.079330 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f4c06-8a2a-41d2-9891-9e9ae8331744 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:38.079638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f4c06-8a2a-41d2-9891-9e9ae8331744 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:38.081036 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f4c06-8a2a-41d2-9891-9e9ae8331744 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:38.088760 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:38.089726 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 221/444] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:38.090305 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f4c06-8a2a-41d2-9891-9e9ae8331744 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:38.136193 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72f6f0-764f-4f96-a442-90dca25ffd79 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:38.170189 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5b329-47f1-45f4-a627-1bd33ebe00e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:38.170910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5b329-47f1-45f4-a627-1bd33ebe00e4 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:38.171192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5b329-47f1-45f4-a627-1bd33ebe00e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:38.171467 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5b329-47f1-45f4-a627-1bd33ebe00e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:38.172319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d5b329-47f1-45f4-a627-1bd33ebe00e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:38.172622 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d5b329-47f1-45f4-a627-1bd33ebe00e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:38.173998 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d5b329-47f1-45f4-a627-1bd33ebe00e4 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:38.181805 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 224/445] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:24:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:38.182575 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5b329-47f1-45f4-a627-1bd33ebe00e4 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:38.189799 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f4c06-8a2a-41d2-9891-9e9ae8331744 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:38.226527 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fe402-87cf-48d6-a455-3b3e80dd4395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:38.227381 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fe402-87cf-48d6-a455-3b3e80dd4395 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:38.227663 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fe402-87cf-48d6-a455-3b3e80dd4395 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:38.227968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fe402-87cf-48d6-a455-3b3e80dd4395 None admin] PATH_INFO: `/v3/groups/62ddfa5186074f6eb9514f62cffa48da` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:38.229144 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85fe402-87cf-48d6-a455-3b3e80dd4395 None admin] RBAC: Authorizing `identity:get_group(group_id=62ddfa5186074f6eb9514f62cffa48da)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:38.230584 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85fe402-87cf-48d6-a455-3b3e80dd4395 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:38.232390 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 222/446] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jun 6 10:24:38 2022] GET /identity/v3/groups/62ddfa5186074f6eb9514f62cffa48da => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:38.232981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fe402-87cf-48d6-a455-3b3e80dd4395 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:38.259456 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:38.296923 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5b329-47f1-45f4-a627-1bd33ebe00e4 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:38.316971 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:38.331722 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4ff94-df4f-44c7-8531-4ba5cad787ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:38.332955 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4ff94-df4f-44c7-8531-4ba5cad787ec None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:38.333801 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4ff94-df4f-44c7-8531-4ba5cad787ec None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:38.334083 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4ff94-df4f-44c7-8531-4ba5cad787ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:38.337456 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fe402-87cf-48d6-a455-3b3e80dd4395 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:38.371559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f6ae2-c07b-4cfc-bcea-34af7e021b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:38.372222 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f6ae2-c07b-4cfc-bcea-34af7e021b59 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:38.372509 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f6ae2-c07b-4cfc-bcea-34af7e021b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:38.372787 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f6ae2-c07b-4cfc-bcea-34af7e021b59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:38.403054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4ff94-df4f-44c7-8531-4ba5cad787ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:38.404615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4ff94-df4f-44c7-8531-4ba5cad787ec None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:38.440675 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f6ae2-c07b-4cfc-bcea-34af7e021b59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:38.442272 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f6ae2-c07b-4cfc-bcea-34af7e021b59 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:38.467673 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:38.499641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 225/447] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:38.500642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4ff94-df4f-44c7-8531-4ba5cad787ec None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:38.524380 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:38.545805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 223/448] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:38.546635 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f6ae2-c07b-4cfc-bcea-34af7e021b59 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:38.601784 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4ff94-df4f-44c7-8531-4ba5cad787ec None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:38.636544 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb53a801-9e93-4185-8cb3-83b0f7bc52f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:38.637203 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53a801-9e93-4185-8cb3-83b0f7bc52f8 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:38.637481 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53a801-9e93-4185-8cb3-83b0f7bc52f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:38.637752 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53a801-9e93-4185-8cb3-83b0f7bc52f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:38.638577 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb53a801-9e93-4185-8cb3-83b0f7bc52f8 None admin] RBAC: Adding query filter params (group.id=62ddfa5186074f6eb9514f62cffa48da, role.id=6bcb06bf4aa4442ebcfebeadf1af41de, scope.project.id=d65d04e96c4b4b87be63be85bb9731a0) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:38.638886 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb53a801-9e93-4185-8cb3-83b0f7bc52f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:38.640348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb53a801-9e93-4185-8cb3-83b0f7bc52f8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:38.655950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 226/449] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jun 6 10:24:38 2022] GET /identity/v3/role_assignments?group.id=62ddfa5186074f6eb9514f62cffa48da&scope.project.id=d65d04e96c4b4b87be63be85bb9731a0&role.id=6bcb06bf4aa4442ebcfebeadf1af41de => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:38.655950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb53a801-9e93-4185-8cb3-83b0f7bc52f8 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:38.656284 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f6ae2-c07b-4cfc-bcea-34af7e021b59 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:38.686677 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:38.697143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf447a9-9e1e-4b2d-b9c8-6cdcf6acb648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:38.698018 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf447a9-9e1e-4b2d-b9c8-6cdcf6acb648 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:38.698301 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf447a9-9e1e-4b2d-b9c8-6cdcf6acb648 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:38.698571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf447a9-9e1e-4b2d-b9c8-6cdcf6acb648 None admin] PATH_INFO: `/v3/projects/863195bccbf14e04a9a61a90f52ba9f1` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:38.699806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf447a9-9e1e-4b2d-b9c8-6cdcf6acb648 None admin] RBAC: Authorizing `identity:get_project(project_id=863195bccbf14e04a9a61a90f52ba9f1)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:38.701274 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf447a9-9e1e-4b2d-b9c8-6cdcf6acb648 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:38.703172 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 224/450] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:38 2022] GET /identity/v3/projects/863195bccbf14e04a9a61a90f52ba9f1 => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:38.709486 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf447a9-9e1e-4b2d-b9c8-6cdcf6acb648 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:38.751516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:38.774440 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb53a801-9e93-4185-8cb3-83b0f7bc52f8 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:38.813958 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e350d-0321-442d-a011-5a67570b8123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:38.814843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e350d-0321-442d-a011-5a67570b8123 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:38.815125 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e350d-0321-442d-a011-5a67570b8123 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:38.815396 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e350d-0321-442d-a011-5a67570b8123 None admin] PATH_INFO: `/v3/projects/d65d04e96c4b4b87be63be85bb9731a0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:38.816767 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e350d-0321-442d-a011-5a67570b8123 None admin] RBAC: Authorizing `identity:get_project(project_id=d65d04e96c4b4b87be63be85bb9731a0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:38.818425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e350d-0321-442d-a011-5a67570b8123 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:38.820231 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 227/451] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:24:38 2022] GET /identity/v3/projects/d65d04e96c4b4b87be63be85bb9731a0 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:38.825880 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e350d-0321-442d-a011-5a67570b8123 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:38.827143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf447a9-9e1e-4b2d-b9c8-6cdcf6acb648 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:38.870786 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e8057-8782-458f-80ec-2a5534cc0a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:38.871572 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e8057-8782-458f-80ec-2a5534cc0a64 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:38.871919 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e8057-8782-458f-80ec-2a5534cc0a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:38.872198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e8057-8782-458f-80ec-2a5534cc0a64 None admin] PATH_INFO: `/v3/groups/83a537bba06a44fbb6d102d17532495d` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:38.873385 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e8057-8782-458f-80ec-2a5534cc0a64 None admin] RBAC: Authorizing `identity:get_group(group_id=83a537bba06a44fbb6d102d17532495d)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:38.875285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e8057-8782-458f-80ec-2a5534cc0a64 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:38.876952 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 225/452] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jun 6 10:24:38 2022] GET /identity/v3/groups/83a537bba06a44fbb6d102d17532495d => generated 229 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:38.883531 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e8057-8782-458f-80ec-2a5534cc0a64 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:38.955757 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e350d-0321-442d-a011-5a67570b8123 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:38.958410 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:38.998148 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edddf76d-7190-4866-868e-a690ac1d2b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:38.999329 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edddf76d-7190-4866-868e-a690ac1d2b43 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:38.999329 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edddf76d-7190-4866-868e-a690ac1d2b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:38.999534 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edddf76d-7190-4866-868e-a690ac1d2b43 None admin] PATH_INFO: `/v3/groups/62ddfa5186074f6eb9514f62cffa48da` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:39.000771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edddf76d-7190-4866-868e-a690ac1d2b43 None admin] RBAC: Authorizing `identity:get_group(group_id=62ddfa5186074f6eb9514f62cffa48da)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:39.002226 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edddf76d-7190-4866-868e-a690ac1d2b43 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:39.004072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 228/453] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jun 6 10:24:38 2022] GET /identity/v3/groups/62ddfa5186074f6eb9514f62cffa48da => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:39.008775 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e8057-8782-458f-80ec-2a5534cc0a64 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:39.009878 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edddf76d-7190-4866-868e-a690ac1d2b43 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:39.034335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:39.055512 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d115114-b54d-4090-8ab4-7caf7b733d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:39.056485 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d115114-b54d-4090-8ab4-7caf7b733d80 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:39.056770 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d115114-b54d-4090-8ab4-7caf7b733d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:39.057046 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d115114-b54d-4090-8ab4-7caf7b733d80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:39.057888 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d115114-b54d-4090-8ab4-7caf7b733d80 None admin] RBAC: Adding query filter params (group.id=83a537bba06a44fbb6d102d17532495d, role.id=0f929204eb7b4ea8859338ed2d3ff75b, scope.project.id=863195bccbf14e04a9a61a90f52ba9f1) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:39.058202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d115114-b54d-4090-8ab4-7caf7b733d80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:39.061489 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d115114-b54d-4090-8ab4-7caf7b733d80 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:39.067232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 226/454] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jun 6 10:24:38 2022] GET /identity/v3/role_assignments?group.id=83a537bba06a44fbb6d102d17532495d&scope.project.id=863195bccbf14e04a9a61a90f52ba9f1&role.id=0f929204eb7b4ea8859338ed2d3ff75b => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:39.123064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edddf76d-7190-4866-868e-a690ac1d2b43 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:39.159836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60938561-ef19-402c-8b35-8c2f27fd687a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:39.161219 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60938561-ef19-402c-8b35-8c2f27fd687a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:39.161503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60938561-ef19-402c-8b35-8c2f27fd687a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:39.161782 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60938561-ef19-402c-8b35-8c2f27fd687a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:39.162635 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60938561-ef19-402c-8b35-8c2f27fd687a None admin] RBAC: Adding query filter params (group.id=62ddfa5186074f6eb9514f62cffa48da, role.id=f214f8dbc3354cfc81fa2c2f116980e0, scope.project.id=d65d04e96c4b4b87be63be85bb9731a0) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:39.162945 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60938561-ef19-402c-8b35-8c2f27fd687a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:39.164589 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60938561-ef19-402c-8b35-8c2f27fd687a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:39.171389 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 229/455] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jun 6 10:24:39 2022] GET /identity/v3/role_assignments?group.id=62ddfa5186074f6eb9514f62cffa48da&scope.project.id=d65d04e96c4b4b87be63be85bb9731a0&role.id=f214f8dbc3354cfc81fa2c2f116980e0 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:41.921518 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030779c-f6c4-4762-98cb-edabb112b498 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:41.922379 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030779c-f6c4-4762-98cb-edabb112b498 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:41.922922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030779c-f6c4-4762-98cb-edabb112b498 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:41.924692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 227/456] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:41.931107 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563034a0-d3b4-42d1-8b20-7e6ad8863585 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:41.932012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563034a0-d3b4-42d1-8b20-7e6ad8863585 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:41.932495 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563034a0-d3b4-42d1-8b20-7e6ad8863585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:41.954701 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-563034a0-d3b4-42d1-8b20-7e6ad8863585 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:41.956314 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-563034a0-d3b4-42d1-8b20-7e6ad8863585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:42.047863 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 230/457] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:42.093709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030779c-f6c4-4762-98cb-edabb112b498 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:42.205301 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2ef20-405c-4a59-be2c-3dec33640d56 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:42.205745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2ef20-405c-4a59-be2c-3dec33640d56 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:42.206253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2ef20-405c-4a59-be2c-3dec33640d56 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:42.210821 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 231/458] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:24:42.217449 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575cf918-a4a7-4879-b871-b870a55a0954 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:42.218011 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575cf918-a4a7-4879-b871-b870a55a0954 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:42.218629 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575cf918-a4a7-4879-b871-b870a55a0954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:42.243578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030779c-f6c4-4762-98cb-edabb112b498 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:42.246363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-575cf918-a4a7-4879-b871-b870a55a0954 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:42.247594 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-575cf918-a4a7-4879-b871-b870a55a0954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:42.272015 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:42.281275 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:42.295144 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435fcb0-4af8-4353-baef-14385023bad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:42.296922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435fcb0-4af8-4353-baef-14385023bad1 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:42.300239 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435fcb0-4af8-4353-baef-14385023bad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:42.301400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435fcb0-4af8-4353-baef-14385023bad1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:42.322072 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435fcb0-4af8-4353-baef-14385023bad1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:42.323538 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435fcb0-4af8-4353-baef-14385023bad1 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:42.362137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-2435fcb0-4af8-4353-baef-14385023bad1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:42.362137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:42.362137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:42.362137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:42.362137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:42.362137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:42.362137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:42.362137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:42.362137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:42.362137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:42.362137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:42.362137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:42.362137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:42.362137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:42.362137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:42.363252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:42.363252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:42.363252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:42.363252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:42.363252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:42.363252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:42.363252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:42.363252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:42.363252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:42.363252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:42.363252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:42.363252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:42.363252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:42.363252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:42.363252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:42.363252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:42.363252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:42.364261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:42.364261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:42.364261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:42.364261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:42.366477 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 228/459] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:42.374318 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435fcb0-4af8-4353-baef-14385023bad1 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:42.444089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 232/460] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 10:24:42.484975 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575cf918-a4a7-4879-b871-b870a55a0954 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:42.497829 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435fcb0-4af8-4353-baef-14385023bad1 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:42.535039 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8817df9-dec8-4688-951e-9099be713571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:42.536558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8817df9-dec8-4688-951e-9099be713571 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:42.536986 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8817df9-dec8-4688-951e-9099be713571 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:42.537463 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8817df9-dec8-4688-951e-9099be713571 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:42.538460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8817df9-dec8-4688-951e-9099be713571 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:42.538946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8817df9-dec8-4688-951e-9099be713571 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:42.540551 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8817df9-dec8-4688-951e-9099be713571 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:42.556973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 229/461] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:42.562825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4585f92-4cab-4e14-8459-c1f9140d9e02 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:42.563365 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4585f92-4cab-4e14-8459-c1f9140d9e02 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:42.563851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4585f92-4cab-4e14-8459-c1f9140d9e02 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:42.565496 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 230/462] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:42.566314 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4585f92-4cab-4e14-8459-c1f9140d9e02 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:42.610512 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575cf918-a4a7-4879-b871-b870a55a0954 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:42.611495 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:42.646704 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa26ab5f-7528-4ebd-bbab-549e5abe28e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:42.648219 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26ab5f-7528-4ebd-bbab-549e5abe28e9 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:42.648505 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26ab5f-7528-4ebd-bbab-549e5abe28e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:42.649037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26ab5f-7528-4ebd-bbab-549e5abe28e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:42.664317 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa26ab5f-7528-4ebd-bbab-549e5abe28e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:42.665782 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa26ab5f-7528-4ebd-bbab-549e5abe28e9 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:42.668470 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:42.684430 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4585f92-4cab-4e14-8459-c1f9140d9e02 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:42.684962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-fa26ab5f-7528-4ebd-bbab-549e5abe28e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:42.684962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:42.684962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:42.684962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:42.684962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:42.684962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:42.684962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:42.684962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:42.684962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:42.684962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:42.684962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:42.684962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:42.684962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:42.684962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:42.684962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:42.685788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:42.685788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:42.685788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:42.685788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:42.685788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:42.685788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:42.685788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:42.685788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:42.685788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:42.685788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:42.685788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:42.685788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:42.685788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:42.685788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:42.685788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:42.685788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:42.685788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:42.686653 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:42.686653 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:42.686653 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:42.686653 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:42.687382 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 233/463] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:42.689824 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e31e4-72d3-4d39-b343-5452608ca431 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:42.690607 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e31e4-72d3-4d39-b343-5452608ca431 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:42.691337 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e31e4-72d3-4d39-b343-5452608ca431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:42.712455 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-332e31e4-72d3-4d39-b343-5452608ca431 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:42.715256 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-332e31e4-72d3-4d39-b343-5452608ca431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:42.728316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0fce2c-f8f5-49cb-a57d-5224ac13b964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:42.728986 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0fce2c-f8f5-49cb-a57d-5224ac13b964 None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:42.729385 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0fce2c-f8f5-49cb-a57d-5224ac13b964 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:42.729668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0fce2c-f8f5-49cb-a57d-5224ac13b964 None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:42.730508 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0fce2c-f8f5-49cb-a57d-5224ac13b964 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:42.731970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0fce2c-f8f5-49cb-a57d-5224ac13b964 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:42.762830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 231/464] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jun 6 10:24:42 2022] POST /identity/v3/users => generated 246 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:24:42.762830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833c893-6168-4f67-82c5-2b9f737e2740 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:42.762830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833c893-6168-4f67-82c5-2b9f737e2740 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:42.762830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833c893-6168-4f67-82c5-2b9f737e2740 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:42.762830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 232/465] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:42.762830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833c893-6168-4f67-82c5-2b9f737e2740 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:42.799688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 234/466] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:42.801068 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fadda3-1c7f-4c57-850e-4fac58edbe7d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:42.801454 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fadda3-1c7f-4c57-850e-4fac58edbe7d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:42.801726 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fadda3-1c7f-4c57-850e-4fac58edbe7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:42.828715 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-d0fadda3-1c7f-4c57-850e-4fac58edbe7d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:42.830838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d0fadda3-1c7f-4c57-850e-4fac58edbe7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:42.929755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:42.942173 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833c893-6168-4f67-82c5-2b9f737e2740 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:42.988215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070fbf53-2b16-45b0-8b55-7f4be8bb9f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:42.989304 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fbf53-2b16-45b0-8b55-7f4be8bb9f84 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:42.989786 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fbf53-2b16-45b0-8b55-7f4be8bb9f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:42.990258 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fbf53-2b16-45b0-8b55-7f4be8bb9f84 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:42.991219 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070fbf53-2b16-45b0-8b55-7f4be8bb9f84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:42.991772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070fbf53-2b16-45b0-8b55-7f4be8bb9f84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:42.993515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070fbf53-2b16-45b0-8b55-7f4be8bb9f84 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:43.015476 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 233/467] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:43.017298 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070fbf53-2b16-45b0-8b55-7f4be8bb9f84 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:43.076017 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 235/468] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:43.076869 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fadda3-1c7f-4c57-850e-4fac58edbe7d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:43.138909 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:43.153512 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070fbf53-2b16-45b0-8b55-7f4be8bb9f84 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:43.196943 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fadda3-1c7f-4c57-850e-4fac58edbe7d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:43.198433 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ba5c3-bf49-4510-8742-0fd4d46ab22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:43.199484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ba5c3-bf49-4510-8742-0fd4d46ab22d None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:43.199860 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ba5c3-bf49-4510-8742-0fd4d46ab22d None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:43.201000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ba5c3-bf49-4510-8742-0fd4d46ab22d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:43.214246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ba5c3-bf49-4510-8742-0fd4d46ab22d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:43.216023 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ba5c3-bf49-4510-8742-0fd4d46ab22d None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:43.234448 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c7310-b3c2-410b-a7da-fc0b8b5536ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:43.235431 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c7310-b3c2-410b-a7da-fc0b8b5536ad None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:43.236004 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c7310-b3c2-410b-a7da-fc0b8b5536ad None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:43.236448 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c7310-b3c2-410b-a7da-fc0b8b5536ad None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:43.237310 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c7310-b3c2-410b-a7da-fc0b8b5536ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:43.239542 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c7310-b3c2-410b-a7da-fc0b8b5536ad None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:43.240190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-7a7ba5c3-bf49-4510-8742-0fd4d46ab22d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:43.240190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:43.240190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:43.240190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:43.240190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:43.240190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:43.240190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:43.240190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:43.240190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:43.240190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:43.240190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:43.240190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:43.240190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:43.240190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:43.240190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:43.241022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:43.241022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:43.241022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:43.241022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:43.241022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:43.241022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:43.241022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:43.241022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:43.241022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:43.241022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:43.241022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:43.241022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:43.241022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:43.241022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:43.241022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:43.241022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:43.241022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:43.241883 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:43.241883 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:43.241883 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:43.241883 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:43.241883 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 234/469] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:43.247394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ba5c3-bf49-4510-8742-0fd4d46ab22d None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:43.258815 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:43.270096 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 236/470] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jun 6 10:24:43 2022] POST /identity/v3/users => generated 248 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:24:43.271289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c7310-b3c2-410b-a7da-fc0b8b5536ad None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:43.405654 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c7310-b3c2-410b-a7da-fc0b8b5536ad None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:43.424212 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:43.451228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ba5c3-bf49-4510-8742-0fd4d46ab22d None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:43.453128 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce808f-f7e2-4b37-b047-ca81bafcb8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:43.454092 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce808f-f7e2-4b37-b047-ca81bafcb8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:43.454585 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce808f-f7e2-4b37-b047-ca81bafcb8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:43.455073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce808f-f7e2-4b37-b047-ca81bafcb8e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:43.456029 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ce808f-f7e2-4b37-b047-ca81bafcb8e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:43.456525 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ce808f-f7e2-4b37-b047-ca81bafcb8e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:43.458215 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ce808f-f7e2-4b37-b047-ca81bafcb8e7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:43.471847 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 237/471] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:43.477874 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce808f-f7e2-4b37-b047-ca81bafcb8e7 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:43.514630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ad1e1-df94-4a41-b5d9-91f3c4f73252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:43.516780 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ad1e1-df94-4a41-b5d9-91f3c4f73252 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:43.517241 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ad1e1-df94-4a41-b5d9-91f3c4f73252 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:43.517684 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ad1e1-df94-4a41-b5d9-91f3c4f73252 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:43.543746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ad1e1-df94-4a41-b5d9-91f3c4f73252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:43.546656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ad1e1-df94-4a41-b5d9-91f3c4f73252 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:43.587243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-2d0ad1e1-df94-4a41-b5d9-91f3c4f73252 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:43.587243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:43.587243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:43.587243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:43.587243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:43.587243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:43.587243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:43.587243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:43.587243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:43.587243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:43.587243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:43.587243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:43.587243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:43.587243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:43.587243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:43.588193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:43.588193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:43.588193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:43.588193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:43.588193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:43.588193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:43.588193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:43.588193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:43.588193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:43.588193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:43.588193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:43.588193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:43.588193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:43.588193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:43.588193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:43.588193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:43.588193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:43.589062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:43.589062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:43.589062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:43.589062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:43.595490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 235/472] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:43.600376 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ad1e1-df94-4a41-b5d9-91f3c4f73252 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:43.661976 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce808f-f7e2-4b37-b047-ca81bafcb8e7 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:43.681460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:43.706714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723a9de-bade-4cf9-9cf6-4cc306cbc0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:43.708459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723a9de-bade-4cf9-9cf6-4cc306cbc0ce None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:43.708883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723a9de-bade-4cf9-9cf6-4cc306cbc0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:43.709463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723a9de-bade-4cf9-9cf6-4cc306cbc0ce None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:43.710696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8723a9de-bade-4cf9-9cf6-4cc306cbc0ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:43.712590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8723a9de-bade-4cf9-9cf6-4cc306cbc0ce None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:43.733430 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ad1e1-df94-4a41-b5d9-91f3c4f73252 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:43.747219 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 238/473] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jun 6 10:24:43 2022] POST /identity/v3/users => generated 248 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:24:43.753621 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906ce94-8585-465e-83af-f90be7e95e85 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:43.754822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906ce94-8585-465e-83af-f90be7e95e85 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:43.755275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906ce94-8585-465e-83af-f90be7e95e85 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:43.757795 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 239/474] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:43.763343 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bb48e-793b-45c0-b9fc-e660c475e48f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:43.763631 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bb48e-793b-45c0-b9fc-e660c475e48f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:43.767416 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bb48e-793b-45c0-b9fc-e660c475e48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:43.785530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d1608-c499-4d33-a351-bcc39a9c6db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:43.786645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d1608-c499-4d33-a351-bcc39a9c6db7 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:43.787128 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d1608-c499-4d33-a351-bcc39a9c6db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:43.787602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d1608-c499-4d33-a351-bcc39a9c6db7 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:43.788583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d1608-c499-4d33-a351-bcc39a9c6db7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:43.789162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d1608-c499-4d33-a351-bcc39a9c6db7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:43.789805 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-498bb48e-793b-45c0-b9fc-e660c475e48f None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:43.791010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-498bb48e-793b-45c0-b9fc-e660c475e48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:43.791355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d1608-c499-4d33-a351-bcc39a9c6db7 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:43.806450 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 236/475] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:43.811486 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d1608-c499-4d33-a351-bcc39a9c6db7 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:43.820644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:43.891241 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 240/476] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:43.936492 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bb48e-793b-45c0-b9fc-e660c475e48f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:43.993268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d1608-c499-4d33-a351-bcc39a9c6db7 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:44.026952 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:44.045633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992f2d0-1519-4efd-8a58-c744c62e2d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:44.046304 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992f2d0-1519-4efd-8a58-c744c62e2d44 None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:44.046596 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992f2d0-1519-4efd-8a58-c744c62e2d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:44.046874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992f2d0-1519-4efd-8a58-c744c62e2d44 None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:44.047583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992f2d0-1519-4efd-8a58-c744c62e2d44 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:44.049048 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992f2d0-1519-4efd-8a58-c744c62e2d44 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:44.075505 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 237/477] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jun 6 10:24:43 2022] POST /identity/v3/users => generated 249 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:24:44.091678 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bb48e-793b-45c0-b9fc-e660c475e48f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:44.132320 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a5dd9e-f2a8-466e-beac-453ef9bd7171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:44.133309 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5dd9e-f2a8-466e-beac-453ef9bd7171 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:44.133793 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5dd9e-f2a8-466e-beac-453ef9bd7171 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:44.134256 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5dd9e-f2a8-466e-beac-453ef9bd7171 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:44.135193 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5dd9e-f2a8-466e-beac-453ef9bd7171 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:44.136948 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5dd9e-f2a8-466e-beac-453ef9bd7171 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:44.147475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.sql.core [None req-a3a5dd9e-f2a8-466e-beac-453ef9bd7171 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 06 10:24:44.147475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 06 10:24:44.147475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [parameters: {'id': '13b0e5feda734b289edc3e148396551b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 06 10:24:44.147475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80240) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 06 10:24:44.165843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-a3a5dd9e-f2a8-466e-beac-453ef9bd7171 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 06 10:24:44.165843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:44.165843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 10:24:44.165843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 10:24:44.165843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 10:24:44.165843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 10:24:44.165843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 10:24:44.165843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 10:24:44.165843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 10:24:44.165843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application conn.query(q) Jun 06 10:24:44.165843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 10:24:44.165843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 10:24:44.165843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 10:24:44.165843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application result.read() Jun 06 10:24:44.165843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 10:24:44.170316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 10:24:44.170316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 10:24:44.170316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 10:24:44.170316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 10:24:44.170316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 10:24:44.170316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 10:24:44.170316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 10:24:44.170316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 06 10:24:44.170316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:44.170316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 06 10:24:44.170316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:44.170316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:44.170316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 06 10:24:44.170316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 06 10:24:44.170316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 06 10:24:44.170316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return ref.to_dict() Jun 06 10:24:44.170316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 10:24:44.170316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application next(self.gen) Jun 06 10:24:44.171260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 06 10:24:44.171260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application yield resource Jun 06 10:24:44.171260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 10:24:44.171260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application next(self.gen) Jun 06 10:24:44.171260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 06 10:24:44.171260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self.session.rollback() Jun 06 10:24:44.171260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 06 10:24:44.171260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self.force_reraise() Jun 06 10:24:44.171260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 06 10:24:44.171260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise self.value Jun 06 10:24:44.171260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 06 10:24:44.171260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 06 10:24:44.171260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 06 10:24:44.171260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application session.commit() Jun 06 10:24:44.171260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 06 10:24:44.171260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 06 10:24:44.171260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 06 10:24:44.172475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._prepare_impl() Jun 06 10:24:44.172475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 06 10:24:44.172475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self.session.flush() Jun 06 10:24:44.172475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 06 10:24:44.172475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._flush(objects) Jun 06 10:24:44.172475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 06 10:24:44.172475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 06 10:24:44.172475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 06 10:24:44.172475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application compat.raise_( Jun 06 10:24:44.172475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 10:24:44.172475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception Jun 06 10:24:44.172475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 06 10:24:44.172475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application flush_context.execute() Jun 06 10:24:44.172475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 06 10:24:44.172475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rec.execute(self) Jun 06 10:24:44.172475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 06 10:24:44.172475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 06 10:24:44.173639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 06 10:24:44.173639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 06 10:24:44.173639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 06 10:24:44.173639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 06 10:24:44.173639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 06 10:24:44.173639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 06 10:24:44.173639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 06 10:24:44.173639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 06 10:24:44.173639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 06 10:24:44.173639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 06 10:24:44.173639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 06 10:24:44.173639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 06 10:24:44.173639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 06 10:24:44.173639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 06 10:24:44.173639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 10:24:44.173639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception Jun 06 10:24:44.176905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 10:24:44.176905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 10:24:44.176905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 10:24:44.176905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 10:24:44.176905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 10:24:44.176905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 10:24:44.176905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 10:24:44.176905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application conn.query(q) Jun 06 10:24:44.176905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 10:24:44.176905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 10:24:44.176905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 10:24:44.176905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application result.read() Jun 06 10:24:44.176905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 10:24:44.176905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 10:24:44.176905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 10:24:44.176905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 10:24:44.178294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 10:24:44.178294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 10:24:44.178294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 10:24:44.178294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 10:24:44.178294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 06 10:24:44.178294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 06 10:24:44.178294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [parameters: {'id': '13b0e5feda734b289edc3e148396551b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 06 10:24:44.178294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 06 10:24:44.178294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:44.178294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:24:44.178294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:44.178294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:44.178294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:44.178294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:44.178294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:44.178294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:44.179733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:44.179733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:44.179733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:44.179733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:44.179733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:44.179733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:44.179733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 06 10:24:44.179733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 06 10:24:44.179733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:44.179733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:44.179733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 06 10:24:44.179733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 06 10:24:44.179733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 06 10:24:44.179733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 06 10:24:44.179733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 06 10:24:44.179733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:44.181132 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 241/478] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Jun 6 10:24:43 2022] POST /identity/v3/roles => generated 144 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:24:44.185823 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992f2d0-1519-4efd-8a58-c744c62e2d44 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:44.307457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992f2d0-1519-4efd-8a58-c744c62e2d44 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:44.348443 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365786bd-1438-456c-89e6-d6cec13ea842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:44.349484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365786bd-1438-456c-89e6-d6cec13ea842 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:44.349911 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365786bd-1438-456c-89e6-d6cec13ea842 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:44.350323 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365786bd-1438-456c-89e6-d6cec13ea842 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:44.361038 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365786bd-1438-456c-89e6-d6cec13ea842 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:44.370608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-365786bd-1438-456c-89e6-d6cec13ea842 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:44.370608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:44.370608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:44.370608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:44.370608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:44.370608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:44.370608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:44.370608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:44.370608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:44.370608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:44.370608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:44.370608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:44.370608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:44.370608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:44.370608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:44.370608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:44.371757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:44.371757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:44.371757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:44.371757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:44.371757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:44.371757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:44.371757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:44.371757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:44.371757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:44.371757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:44.371757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:44.371757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:44.371757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:44.371757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:44.371757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:44.371757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:44.372782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:44.372782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:44.372782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:44.372782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:44.372782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:44.372782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:44.372782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:44.372782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:44.372782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:44.372782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:44.372782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 10:24:44.372782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:44.374912 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 238/479] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:24:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:24:44.391249 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a5dd9e-f2a8-466e-beac-453ef9bd7171 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:44.433105 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:44.518443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a5dd9e-f2a8-466e-beac-453ef9bd7171 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:44.554923 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb774d9-7567-4da7-9c3f-36ebc21a6f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:44.555960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb774d9-7567-4da7-9c3f-36ebc21a6f3d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:44.556422 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb774d9-7567-4da7-9c3f-36ebc21a6f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:44.556872 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb774d9-7567-4da7-9c3f-36ebc21a6f3d None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:44.557794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb774d9-7567-4da7-9c3f-36ebc21a6f3d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:44.558327 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb774d9-7567-4da7-9c3f-36ebc21a6f3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:44.559984 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb774d9-7567-4da7-9c3f-36ebc21a6f3d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:44.569700 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 242/480] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:24:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:45.487393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbaac6-d49e-4faa-8186-6069bf94ed26 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:45.487913 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbaac6-d49e-4faa-8186-6069bf94ed26 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:45.487987 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbaac6-d49e-4faa-8186-6069bf94ed26 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:45.489769 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 239/481] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:45.495020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa07ae-706d-402e-a301-f31aab3cac4f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:45.495399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa07ae-706d-402e-a301-f31aab3cac4f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:45.495649 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa07ae-706d-402e-a301-f31aab3cac4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:45.521120 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-fffa07ae-706d-402e-a301-f31aab3cac4f None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:45.522321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fffa07ae-706d-402e-a301-f31aab3cac4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:45.607918 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 243/482] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:45.643438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbaac6-d49e-4faa-8186-6069bf94ed26 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:45.692085 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:45.789466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbaac6-d49e-4faa-8186-6069bf94ed26 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:45.830524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183184e-40f4-423e-a187-e94712804f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:45.831325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183184e-40f4-423e-a187-e94712804f48 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:45.831605 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183184e-40f4-423e-a187-e94712804f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:45.831910 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183184e-40f4-423e-a187-e94712804f48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:45.850667 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183184e-40f4-423e-a187-e94712804f48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:45.858524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-5183184e-40f4-423e-a187-e94712804f48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:45.858524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:45.858524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:45.858524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:45.858524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:45.858524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:45.858524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:45.858524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:45.858524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:45.858524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:45.858524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:45.858524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:45.858524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:45.858524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:45.858524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:45.858524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:45.859387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:45.859387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:45.859387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:45.859387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:45.859387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:45.859387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:45.859387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:45.859387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:45.859387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:45.859387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:45.859387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:45.859387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:45.859387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:45.859387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:45.859387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:45.859387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:45.860235 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:45.860235 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:45.860235 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:45.860235 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:45.860235 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:45.860235 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:45.860235 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:45.860235 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:45.860235 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:45.860235 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:45.860235 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:45.860235 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:45.864615 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 240/483] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:24:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:45.868904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa07ae-706d-402e-a301-f31aab3cac4f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:45.899998 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:46.043483 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa07ae-706d-402e-a301-f31aab3cac4f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:46.088946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf2af4-0930-419e-b8fe-c80fca3eeb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:46.089961 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf2af4-0930-419e-b8fe-c80fca3eeb50 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:46.090437 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf2af4-0930-419e-b8fe-c80fca3eeb50 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:46.090902 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf2af4-0930-419e-b8fe-c80fca3eeb50 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:46.091872 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf2af4-0930-419e-b8fe-c80fca3eeb50 None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:46.092391 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf2af4-0930-419e-b8fe-c80fca3eeb50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:46.094058 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf2af4-0930-419e-b8fe-c80fca3eeb50 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:46.107359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 244/484] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jun 6 10:24:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:46.112919 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183184e-40f4-423e-a187-e94712804f48 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:46.261139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:46.271024 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183184e-40f4-423e-a187-e94712804f48 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:46.292699 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50b090-f405-4f64-989d-6a3d617b4d12 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:46.293072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50b090-f405-4f64-989d-6a3d617b4d12 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:46.293441 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50b090-f405-4f64-989d-6a3d617b4d12 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:46.296253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 245/485] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:46.300421 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8d751-42c6-44d5-a8ea-12ea13a05410 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:46.300743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8d751-42c6-44d5-a8ea-12ea13a05410 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:46.301046 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8d751-42c6-44d5-a8ea-12ea13a05410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:46.315409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28a53b-115a-4c6c-a192-6daa33b18d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:46.316254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28a53b-115a-4c6c-a192-6daa33b18d9a None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:46.317226 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28a53b-115a-4c6c-a192-6daa33b18d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:46.317501 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28a53b-115a-4c6c-a192-6daa33b18d9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:46.338468 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28a53b-115a-4c6c-a192-6daa33b18d9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:46.338862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-1ad8d751-42c6-44d5-a8ea-12ea13a05410 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:46.339988 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28a53b-115a-4c6c-a192-6daa33b18d9a None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:46.340830 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1ad8d751-42c6-44d5-a8ea-12ea13a05410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:46.358893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-fa28a53b-115a-4c6c-a192-6daa33b18d9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:46.358893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:46.358893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:46.358893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:46.358893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:46.358893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:46.358893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:46.358893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:46.358893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:46.358893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:46.358893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:46.358893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:46.358893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:46.358893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:46.358893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:46.359839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:46.359839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:46.359839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:46.359839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:46.359839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:46.359839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:46.359839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:46.359839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:46.359839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:46.359839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:46.359839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:46.359839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:46.359839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:46.359839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:46.359839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:46.359839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:46.359839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:46.360795 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:46.360795 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:46.360795 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:46.360795 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:46.360795 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 241/486] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:46.368992 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28a53b-115a-4c6c-a192-6daa33b18d9a None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:46.482131 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 246/487] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 10:24:46.529375 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8d751-42c6-44d5-a8ea-12ea13a05410 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:46.545581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28a53b-115a-4c6c-a192-6daa33b18d9a None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:46.546082 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:46.582176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089dfb3-c2c1-4df2-866b-9aacf15256e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:46.582851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089dfb3-c2c1-4df2-866b-9aacf15256e3 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:46.583185 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089dfb3-c2c1-4df2-866b-9aacf15256e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:46.583518 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089dfb3-c2c1-4df2-866b-9aacf15256e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:46.584619 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089dfb3-c2c1-4df2-866b-9aacf15256e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:46.586059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089dfb3-c2c1-4df2-866b-9aacf15256e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:46.587450 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089dfb3-c2c1-4df2-866b-9aacf15256e3 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:46.600815 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 242/488] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:46.606233 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089dfb3-c2c1-4df2-866b-9aacf15256e3 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:46.615524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:46.709707 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089dfb3-c2c1-4df2-866b-9aacf15256e3 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:46.749529 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75c8ff-0adb-430c-b35c-403b60ae8bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:46.750327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75c8ff-0adb-430c-b35c-403b60ae8bea None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:46.750606 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75c8ff-0adb-430c-b35c-403b60ae8bea None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:46.750875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75c8ff-0adb-430c-b35c-403b60ae8bea None admin] PATH_INFO: `/v3/users/nova` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:46.752964 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8d751-42c6-44d5-a8ea-12ea13a05410 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:46.760216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75c8ff-0adb-430c-b35c-403b60ae8bea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:46.761864 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75c8ff-0adb-430c-b35c-403b60ae8bea None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:46.793003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-db75c8ff-0adb-430c-b35c-403b60ae8bea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 10:24:46.793003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:46.793003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:46.793003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:46.793003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:46.793003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:46.793003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:46.793003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:46.793003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:46.793003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:46.793003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:46.793003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:46.793003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:24:46.793003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:24:46.793003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:24:46.793003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:24:46.793924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:46.793924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:46.793924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:24:46.793924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:24:46.793924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:24:46.793924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:24:46.793924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:46.793924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:46.793924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:46.793924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:46.793924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:46.793924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:46.793924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:46.793924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:46.793924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:46.793924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:46.793924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:46.793924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:46.795319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:46.795319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:46.795319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:24:46.795319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:24:46.795319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:24:46.795319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:24:46.795319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:24:46.795319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:24:46.795319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 10:24:46.795319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:46.795319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 243/489] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jun 6 10:24:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:24:46.800265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88772200-918c-47e3-84e8-9bccc09a7679 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:46.801467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88772200-918c-47e3-84e8-9bccc09a7679 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:46.801675 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88772200-918c-47e3-84e8-9bccc09a7679 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:46.803175 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 244/490] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:46.804030 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88772200-918c-47e3-84e8-9bccc09a7679 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:46.815631 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32280505-d85a-44dc-9afb-9e294ff5c3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:46.816752 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32280505-d85a-44dc-9afb-9e294ff5c3af None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:46.817041 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32280505-d85a-44dc-9afb-9e294ff5c3af None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:46.817315 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32280505-d85a-44dc-9afb-9e294ff5c3af None admin] PATH_INFO: `/v3/roles/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:46.828396 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32280505-d85a-44dc-9afb-9e294ff5c3af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:46.837400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-32280505-d85a-44dc-9afb-9e294ff5c3af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:46.837400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:46.837400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:46.837400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:46.837400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:46.837400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:46.837400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:46.837400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:46.837400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:46.837400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:46.837400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:46.837400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:46.837400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:46.837400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:46.837400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:46.837400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:46.839307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:46.839307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:46.839307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:46.839307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:46.839307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:46.839307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:46.839307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:46.839307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:46.839307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:46.839307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:46.839307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:46.839307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:46.839307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:46.839307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:46.839307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:46.839307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:46.840996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:46.840996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:46.840996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:46.840996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:46.840996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:46.840996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:46.840996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:46.840996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:46.840996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:46.840996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:46.840996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:46.840996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:46.840996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 247/491] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:24:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:46.840996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86152738-354e-4cbb-bbd7-337fed46f13c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:46.842216 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86152738-354e-4cbb-bbd7-337fed46f13c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:46.842216 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86152738-354e-4cbb-bbd7-337fed46f13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:46.858766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-86152738-354e-4cbb-bbd7-337fed46f13c None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:46.860184 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-86152738-354e-4cbb-bbd7-337fed46f13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:46.877287 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:46.915777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88772200-918c-47e3-84e8-9bccc09a7679 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:46.924777 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:46.940028 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 248/492] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:46.941180 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86152738-354e-4cbb-bbd7-337fed46f13c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:46.954914 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa662f3-70c0-4785-ab86-f50b2909a622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:46.955673 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa662f3-70c0-4785-ab86-f50b2909a622 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:46.956103 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa662f3-70c0-4785-ab86-f50b2909a622 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:46.956446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa662f3-70c0-4785-ab86-f50b2909a622 None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:46.957879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa662f3-70c0-4785-ab86-f50b2909a622 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:46.958784 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa662f3-70c0-4785-ab86-f50b2909a622 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:46.960246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa662f3-70c0-4785-ab86-f50b2909a622 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:47.008580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 245/493] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jun 6 10:24:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:47.009533 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa662f3-70c0-4785-ab86-f50b2909a622 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:47.067899 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86152738-354e-4cbb-bbd7-337fed46f13c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:47.103487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b451484d-67d5-49f6-869d-dc43db218c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:47.104487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451484d-67d5-49f6-869d-dc43db218c49 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:47.104948 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451484d-67d5-49f6-869d-dc43db218c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:47.105429 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451484d-67d5-49f6-869d-dc43db218c49 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:47.106396 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b451484d-67d5-49f6-869d-dc43db218c49 None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:47.106992 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b451484d-67d5-49f6-869d-dc43db218c49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:47.108614 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b451484d-67d5-49f6-869d-dc43db218c49 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:47.118140 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 249/494] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jun 6 10:24:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:47.119174 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b451484d-67d5-49f6-869d-dc43db218c49 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:47.123032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa662f3-70c0-4785-ab86-f50b2909a622 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:47.147933 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:47.160422 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36470e3a-0f26-451e-a30e-4013b17a2683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:47.161240 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36470e3a-0f26-451e-a30e-4013b17a2683 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:47.161522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36470e3a-0f26-451e-a30e-4013b17a2683 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:47.161792 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36470e3a-0f26-451e-a30e-4013b17a2683 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:47.173637 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36470e3a-0f26-451e-a30e-4013b17a2683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:47.180333 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:47.182032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-36470e3a-0f26-451e-a30e-4013b17a2683 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:47.182032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:47.182032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:47.182032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:47.182032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:47.182032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:47.182032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:47.182032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:47.182032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:47.182032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:47.182032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:47.182032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:47.182032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:47.182032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:47.182032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:47.182032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:47.183436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:47.183436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:47.183436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:47.183436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:47.183436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:47.183436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:47.183436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:47.183436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:47.183436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:47.183436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:47.183436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:47.183436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:47.183436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:47.183436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:47.183436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:47.183436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:47.184741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:47.184741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:47.184741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:47.184741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:47.184741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:47.184741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:47.184741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:47.184741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:47.184741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:47.184741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:47.184741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:47.184741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:47.184741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 246/495] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:24:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:47.184741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92e638-fef2-4b9d-9005-5caef9871bcd None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:47.185445 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92e638-fef2-4b9d-9005-5caef9871bcd None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:47.185445 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92e638-fef2-4b9d-9005-5caef9871bcd None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:47.190641 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 247/496] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:47.190641 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92e638-fef2-4b9d-9005-5caef9871bcd None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:47.233444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b451484d-67d5-49f6-869d-dc43db218c49 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:47.268642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549901d-a4f0-45c4-a07a-c7ec2cf23739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:47.269295 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549901d-a4f0-45c4-a07a-c7ec2cf23739 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:47.269609 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549901d-a4f0-45c4-a07a-c7ec2cf23739 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:47.270039 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549901d-a4f0-45c4-a07a-c7ec2cf23739 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:47.300082 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92e638-fef2-4b9d-9005-5caef9871bcd None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:47.337297 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49d5aa-2d43-4fc8-915d-e51aeaa5eef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:47.338013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49d5aa-2d43-4fc8-915d-e51aeaa5eef2 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:47.338285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49d5aa-2d43-4fc8-915d-e51aeaa5eef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:47.338558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49d5aa-2d43-4fc8-915d-e51aeaa5eef2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:47.343003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549901d-a4f0-45c4-a07a-c7ec2cf23739 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:47.344687 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549901d-a4f0-45c4-a07a-c7ec2cf23739 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:47.351869 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d49d5aa-2d43-4fc8-915d-e51aeaa5eef2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:47.353382 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d49d5aa-2d43-4fc8-915d-e51aeaa5eef2 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:47.375078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-7d49d5aa-2d43-4fc8-915d-e51aeaa5eef2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:47.375078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:47.375078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:47.375078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:47.375078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:47.375078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:47.375078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:47.375078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:47.375078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:47.375078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:47.375078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:47.375078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:47.375078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:47.375078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:47.375078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:47.376246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:47.376246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:47.376246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:47.376246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:47.376246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:47.376246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:47.376246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:47.376246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:47.376246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:47.376246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:47.376246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:47.376246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:47.376246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:47.376246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:47.376246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:47.376246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:47.376246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:47.377105 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:47.377105 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:47.377105 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:47.377105 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:47.377105 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 248/497] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:47.377105 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49d5aa-2d43-4fc8-915d-e51aeaa5eef2 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:47.439060 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:47.454213 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:47.516282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 250/498] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:47.518628 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a41386-fa37-4f19-a390-d54160ff48bc None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:47.519408 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a41386-fa37-4f19-a390-d54160ff48bc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:47.520442 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a41386-fa37-4f19-a390-d54160ff48bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:47.525848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49d5aa-2d43-4fc8-915d-e51aeaa5eef2 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:47.545834 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-08a41386-fa37-4f19-a390-d54160ff48bc None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:47.547674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-08a41386-fa37-4f19-a390-d54160ff48bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:47.573016 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96605fe3-e697-492a-9412-4842946db659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:47.573689 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96605fe3-e697-492a-9412-4842946db659 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:47.573970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96605fe3-e697-492a-9412-4842946db659 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:47.574241 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96605fe3-e697-492a-9412-4842946db659 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:47.574990 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96605fe3-e697-492a-9412-4842946db659 None admin] RBAC: Adding query filter params (name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:47.575297 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96605fe3-e697-492a-9412-4842946db659 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:47.577126 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96605fe3-e697-492a-9412-4842946db659 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:47.591871 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 249/499] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jun 6 10:24:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:47.591871 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96605fe3-e697-492a-9412-4842946db659 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:47.639908 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 251/500] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:47.652748 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a41386-fa37-4f19-a390-d54160ff48bc None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:47.732228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:47.736516 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96605fe3-e697-492a-9412-4842946db659 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:47.755722 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:47.767901 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a41386-fa37-4f19-a390-d54160ff48bc None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:47.771458 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f107a-38ad-44b1-aaab-094f5fed490f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:47.772144 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f107a-38ad-44b1-aaab-094f5fed490f None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:47.772428 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f107a-38ad-44b1-aaab-094f5fed490f None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:47.772704 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f107a-38ad-44b1-aaab-094f5fed490f None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:47.773416 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f107a-38ad-44b1-aaab-094f5fed490f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:47.773978 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f107a-38ad-44b1-aaab-094f5fed490f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:47.775359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f107a-38ad-44b1-aaab-094f5fed490f None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:47.787516 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 250/501] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:47.789265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f107a-38ad-44b1-aaab-094f5fed490f None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:47.802420 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f004df9-9f9b-4e8e-8457-94177d0fdae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:47.803211 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f004df9-9f9b-4e8e-8457-94177d0fdae9 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:47.803489 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f004df9-9f9b-4e8e-8457-94177d0fdae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:47.803759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f004df9-9f9b-4e8e-8457-94177d0fdae9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:47.813453 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f004df9-9f9b-4e8e-8457-94177d0fdae9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:47.814874 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f004df9-9f9b-4e8e-8457-94177d0fdae9 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:47.832359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-0f004df9-9f9b-4e8e-8457-94177d0fdae9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:47.832359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:47.832359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:47.832359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:47.832359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:47.832359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:47.832359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:47.832359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:47.832359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:47.832359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:47.832359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:47.832359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:47.832359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:47.832359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:47.832359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:47.833234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:47.833234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:47.833234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:47.833234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:47.833234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:47.833234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:47.833234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:47.833234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:47.833234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:47.833234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:47.833234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:47.833234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:47.833234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:47.833234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:47.833234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:47.833234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:47.833234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:47.834093 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:47.834093 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:47.834093 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:47.834093 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:47.834093 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 252/502] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:47.834357 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f004df9-9f9b-4e8e-8457-94177d0fdae9 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:47.893726 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f107a-38ad-44b1-aaab-094f5fed490f None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:47.928226 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e8add-cfe8-4a62-9323-f92ea0b708b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:47.929002 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e8add-cfe8-4a62-9323-f92ea0b708b3 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:47.929286 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e8add-cfe8-4a62-9323-f92ea0b708b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:47.929560 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e8add-cfe8-4a62-9323-f92ea0b708b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:47.931496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f004df9-9f9b-4e8e-8457-94177d0fdae9 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:47.939319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e8add-cfe8-4a62-9323-f92ea0b708b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:47.940787 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e8add-cfe8-4a62-9323-f92ea0b708b3 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:47.958589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-0b8e8add-cfe8-4a62-9323-f92ea0b708b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:47.958589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:47.958589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:47.958589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:47.958589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:47.958589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:47.958589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:47.958589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:47.958589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:47.958589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:47.958589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:47.958589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:47.958589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:47.958589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:47.958589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:47.959426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:47.959426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:47.959426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:47.959426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:47.959426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:47.959426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:47.959426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:47.959426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:47.959426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:47.959426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:47.959426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:47.959426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:47.959426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:47.959426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:47.959426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:47.959426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:47.959426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:47.964962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:47.964962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:47.964962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:47.964962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:47.964962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 251/503] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:47.964962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300ae2b-b0b3-4b97-9bf9-6eb4cd0b4bf3 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:47.964962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300ae2b-b0b3-4b97-9bf9-6eb4cd0b4bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:47.964962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300ae2b-b0b3-4b97-9bf9-6eb4cd0b4bf3 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:47.964962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 252/504] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:47.965435 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300ae2b-b0b3-4b97-9bf9-6eb4cd0b4bf3 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:47.967613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefec7f0-155b-4b93-9267-5d66876117e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:47.968449 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefec7f0-155b-4b93-9267-5d66876117e0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:47.968736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefec7f0-155b-4b93-9267-5d66876117e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:47.969014 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefec7f0-155b-4b93-9267-5d66876117e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:47.976834 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:47.978453 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:47.980882 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefec7f0-155b-4b93-9267-5d66876117e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:47.989605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-fefec7f0-155b-4b93-9267-5d66876117e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:47.989605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:47.989605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:47.989605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:47.989605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:47.989605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:47.989605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:47.989605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:47.989605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:47.989605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:47.989605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:47.989605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:47.989605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:47.989605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:47.989605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:47.989605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:47.990516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:47.990516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:47.990516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:47.990516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:47.990516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:47.990516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:47.990516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:47.990516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:47.990516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:47.990516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:47.990516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:47.990516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:47.990516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:47.990516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:47.990516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:47.990516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:47.991298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:47.991298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:47.991298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:47.991298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:47.991298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:47.991298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:47.991298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:47.991298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:47.991298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:47.991298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:47.991298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:47.991298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:47.991298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 253/505] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:24:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:47.992644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefec7f0-155b-4b93-9267-5d66876117e0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:48.070451 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300ae2b-b0b3-4b97-9bf9-6eb4cd0b4bf3 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:48.094572 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefec7f0-155b-4b93-9267-5d66876117e0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:48.102738 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198ec05-ad57-4f5c-8896-b6480285000f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:48.103524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198ec05-ad57-4f5c-8896-b6480285000f None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:48.103832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198ec05-ad57-4f5c-8896-b6480285000f None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:48.104110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198ec05-ad57-4f5c-8896-b6480285000f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:48.111287 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d198ec05-ad57-4f5c-8896-b6480285000f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:48.112737 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d198ec05-ad57-4f5c-8896-b6480285000f None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:48.127099 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b9d08-3f66-483d-8962-d64c34b1af90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:48.127300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-d198ec05-ad57-4f5c-8896-b6480285000f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 10:24:48.127300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:48.127300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:48.127300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:48.127300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:48.127300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:48.127300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:48.127300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:48.127300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:48.127300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:48.127300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:48.127300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:48.127300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:24:48.127300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:24:48.127300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:24:48.127300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:24:48.128389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:48.128389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:48.128389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:24:48.128389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:24:48.128389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:24:48.128389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:24:48.128389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:48.128389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:48.128389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:48.128389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:48.128389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:48.128389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:48.128389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:48.128389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:48.128389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:48.128389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:48.128389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:48.128389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:48.129898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:48.129898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:48.129898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:24:48.129898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:24:48.129898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:24:48.129898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:24:48.129898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:24:48.129898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:24:48.129898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 10:24:48.129898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:48.130405 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b9d08-3f66-483d-8962-d64c34b1af90 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:48.130405 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b9d08-3f66-483d-8962-d64c34b1af90 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:48.130405 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b9d08-3f66-483d-8962-d64c34b1af90 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:48.130581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 253/506] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:48.130653 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b9d08-3f66-483d-8962-d64c34b1af90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:48.130719 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198ec05-ad57-4f5c-8896-b6480285000f None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:48.130783 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b9d08-3f66-483d-8962-d64c34b1af90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:48.131306 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b9d08-3f66-483d-8962-d64c34b1af90 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:48.142980 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 254/507] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:48.144277 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695342a4-fe4f-464b-b6bd-d46179f6b932 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:48.144499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695342a4-fe4f-464b-b6bd-d46179f6b932 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:48.144777 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695342a4-fe4f-464b-b6bd-d46179f6b932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:48.160338 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-695342a4-fe4f-464b-b6bd-d46179f6b932 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:48.161523 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-695342a4-fe4f-464b-b6bd-d46179f6b932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:48.206548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:48.217053 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:48.229433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 255/508] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:48.230229 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695342a4-fe4f-464b-b6bd-d46179f6b932 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:48.239673 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198ec05-ad57-4f5c-8896-b6480285000f None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:48.275914 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44923824-bf35-4f53-9bee-233a50ab9e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:48.276566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44923824-bf35-4f53-9bee-233a50ab9e07 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:48.276853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44923824-bf35-4f53-9bee-233a50ab9e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:48.277126 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44923824-bf35-4f53-9bee-233a50ab9e07 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:48.277843 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44923824-bf35-4f53-9bee-233a50ab9e07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:48.278147 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44923824-bf35-4f53-9bee-233a50ab9e07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:48.279514 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44923824-bf35-4f53-9bee-233a50ab9e07 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:48.291105 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 254/509] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:48.292104 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44923824-bf35-4f53-9bee-233a50ab9e07 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:48.328657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695342a4-fe4f-464b-b6bd-d46179f6b932 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:48.362169 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029bc19-3e7e-4db4-b8e3-17f472e500fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:48.362840 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029bc19-3e7e-4db4-b8e3-17f472e500fe None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:48.363142 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029bc19-3e7e-4db4-b8e3-17f472e500fe None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:48.363418 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029bc19-3e7e-4db4-b8e3-17f472e500fe None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:48.364285 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029bc19-3e7e-4db4-b8e3-17f472e500fe None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:48.364594 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029bc19-3e7e-4db4-b8e3-17f472e500fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:48.365961 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029bc19-3e7e-4db4-b8e3-17f472e500fe None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:48.373717 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 256/510] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jun 6 10:24:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:48.374492 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029bc19-3e7e-4db4-b8e3-17f472e500fe None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:48.400358 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44923824-bf35-4f53-9bee-233a50ab9e07 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:48.427467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:48.428038 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:48.433678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789d376-2572-4d1e-8443-b33bb492bede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:48.434341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789d376-2572-4d1e-8443-b33bb492bede None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:48.434629 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789d376-2572-4d1e-8443-b33bb492bede None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:48.434904 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789d376-2572-4d1e-8443-b33bb492bede None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:48.435736 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789d376-2572-4d1e-8443-b33bb492bede None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:48.436075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789d376-2572-4d1e-8443-b33bb492bede None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:48.437510 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789d376-2572-4d1e-8443-b33bb492bede None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:48.454700 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 255/511] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Jun 6 10:24:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:48.455495 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789d376-2572-4d1e-8443-b33bb492bede None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:48.475209 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029bc19-3e7e-4db4-b8e3-17f472e500fe None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:48.508320 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbcd9c0-2751-4062-89d8-d5bcee797f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:48.509110 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcd9c0-2751-4062-89d8-d5bcee797f43 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:48.509392 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcd9c0-2751-4062-89d8-d5bcee797f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:48.509671 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcd9c0-2751-4062-89d8-d5bcee797f43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:48.516223 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbcd9c0-2751-4062-89d8-d5bcee797f43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:48.517664 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbcd9c0-2751-4062-89d8-d5bcee797f43 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:48.533012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-8cbcd9c0-2751-4062-89d8-d5bcee797f43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:24:48.533012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:48.533012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:48.533012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:48.533012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:48.533012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:48.533012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:48.533012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:48.533012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:48.533012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:48.533012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:48.533012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:48.533012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:24:48.533012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:24:48.533012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:24:48.534043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:24:48.534043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:48.534043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:48.534043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:48.534043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:48.534043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:48.534043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:48.534043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:48.534043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:48.534043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:48.534043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:48.534043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:48.534043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:48.534043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:48.534043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:48.534043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:48.534043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:48.535476 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:24:48.535476 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:24:48.535476 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:24:48.535476 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:24:48.535476 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:24:48.535476 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:24:48.535476 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:24:48.535476 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:48.535476 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 257/512] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:24:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:48.535476 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbcd9c0-2751-4062-89d8-d5bcee797f43 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:48.558405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789d376-2572-4d1e-8443-b33bb492bede None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:48.593587 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef865ec-ea17-44f7-a7aa-9680befcc34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:48.594374 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef865ec-ea17-44f7-a7aa-9680befcc34d None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:48.594655 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef865ec-ea17-44f7-a7aa-9680befcc34d None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:48.594929 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef865ec-ea17-44f7-a7aa-9680befcc34d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:48.605045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef865ec-ea17-44f7-a7aa-9680befcc34d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:48.606473 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef865ec-ea17-44f7-a7aa-9680befcc34d None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:48.624137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-5ef865ec-ea17-44f7-a7aa-9680befcc34d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:48.624137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:48.624137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:48.624137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:48.624137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:48.624137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:48.624137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:48.624137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:48.624137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:48.624137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:48.624137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:48.624137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:48.624137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:48.624137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:48.624137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:48.625007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:48.625007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:48.625007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:48.625007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:48.625007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:48.625007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:48.625007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:48.625007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:48.625007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:48.625007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:48.625007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:48.625007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:48.625007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:48.625007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:48.625007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:48.625007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:48.625007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:48.625860 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:48.625860 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:48.625860 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:48.625860 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:48.625860 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 256/513] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:48.626120 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef865ec-ea17-44f7-a7aa-9680befcc34d None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:48.638724 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbcd9c0-2751-4062-89d8-d5bcee797f43 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:48.652573 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:48.668161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:48.675970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdb6cb-4ddb-4ae7-a58f-739c7ef6bc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:48.676805 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdb6cb-4ddb-4ae7-a58f-739c7ef6bc7c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:48.677084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdb6cb-4ddb-4ae7-a58f-739c7ef6bc7c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:48.677357 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdb6cb-4ddb-4ae7-a58f-739c7ef6bc7c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:48.684259 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afdb6cb-4ddb-4ae7-a58f-739c7ef6bc7c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:48.685705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afdb6cb-4ddb-4ae7-a58f-739c7ef6bc7c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:48.701524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-1afdb6cb-4ddb-4ae7-a58f-739c7ef6bc7c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 06 10:24:48.701524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:48.701524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:48.701524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:48.701524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:48.701524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:48.701524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:48.701524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:48.701524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:48.701524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:48.701524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:48.701524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:48.701524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:24:48.701524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:24:48.701524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:24:48.701524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:24:48.702518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:48.702518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:48.702518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:24:48.702518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:24:48.702518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:24:48.702518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:24:48.702518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:48.702518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:48.702518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:48.702518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:48.702518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:48.702518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:48.702518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:48.702518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:48.702518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:48.702518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:48.702518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:48.702518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:48.703413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:48.703413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:48.703413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:24:48.703413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:24:48.703413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:24:48.703413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:24:48.703413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:24:48.703413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:24:48.703413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 06 10:24:48.703413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:48.703413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 258/514] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:48.704017 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdb6cb-4ddb-4ae7-a58f-739c7ef6bc7c None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:48.728946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef865ec-ea17-44f7-a7aa-9680befcc34d None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:48.761703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716dea8b-5cb4-469d-a631-69d47690fece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:48.762485 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716dea8b-5cb4-469d-a631-69d47690fece None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:48.762762 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716dea8b-5cb4-469d-a631-69d47690fece None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:48.763031 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716dea8b-5cb4-469d-a631-69d47690fece None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:48.773486 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716dea8b-5cb4-469d-a631-69d47690fece None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:48.775201 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716dea8b-5cb4-469d-a631-69d47690fece None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:48.793270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-716dea8b-5cb4-469d-a631-69d47690fece None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:48.793270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:48.793270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:48.793270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:48.793270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:48.793270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:48.793270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:48.793270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:48.793270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:48.793270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:48.793270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:48.793270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:48.793270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:48.793270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:48.793270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:48.794139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:48.794139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:48.794139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:48.794139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:48.794139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:48.794139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:48.794139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:48.794139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:48.794139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:48.794139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:48.794139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:48.794139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:48.794139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:48.794139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:48.794139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:48.794139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:48.794139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:48.794969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:48.794969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:48.794969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:48.794969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:48.794969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 257/515] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:48.795224 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716dea8b-5cb4-469d-a631-69d47690fece None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:48.812999 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdb6cb-4ddb-4ae7-a58f-739c7ef6bc7c None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:48.849183 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30aac6-f4b4-46e8-8b87-3f3336781dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:48.849822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30aac6-f4b4-46e8-8b87-3f3336781dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:48.850104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30aac6-f4b4-46e8-8b87-3f3336781dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:48.850381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30aac6-f4b4-46e8-8b87-3f3336781dd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:48.887500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:48.890094 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:48.897545 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716dea8b-5cb4-469d-a631-69d47690fece None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:48.917946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30aac6-f4b4-46e8-8b87-3f3336781dd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:48.919347 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30aac6-f4b4-46e8-8b87-3f3336781dd9 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:48.931705 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ceab7e-7d59-4e27-b29d-1df67ebfc22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:48.932411 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ceab7e-7d59-4e27-b29d-1df67ebfc22c None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:48.932696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ceab7e-7d59-4e27-b29d-1df67ebfc22c None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:48.932973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ceab7e-7d59-4e27-b29d-1df67ebfc22c None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:48.933754 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ceab7e-7d59-4e27-b29d-1df67ebfc22c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:48.934063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ceab7e-7d59-4e27-b29d-1df67ebfc22c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:48.935515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ceab7e-7d59-4e27-b29d-1df67ebfc22c None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:48.952971 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 258/516] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:24:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:48.953766 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ceab7e-7d59-4e27-b29d-1df67ebfc22c None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:49.016698 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 259/517] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:49.017384 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30aac6-f4b4-46e8-8b87-3f3336781dd9 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:49.059045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ceab7e-7d59-4e27-b29d-1df67ebfc22c None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:49.091366 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:49.095508 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b270f-7610-40f5-a38f-f72667d89fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:49.096415 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b270f-7610-40f5-a38f-f72667d89fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:49.096897 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b270f-7610-40f5-a38f-f72667d89fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:49.097384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b270f-7610-40f5-a38f-f72667d89fe2 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:49.098771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b270f-7610-40f5-a38f-f72667d89fe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:49.099306 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b270f-7610-40f5-a38f-f72667d89fe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:49.101563 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b270f-7610-40f5-a38f-f72667d89fe2 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:49.117030 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 259/518] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:49.117932 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b270f-7610-40f5-a38f-f72667d89fe2 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:49.127435 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30aac6-f4b4-46e8-8b87-3f3336781dd9 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:49.136192 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:49.162344 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86777a3f-458d-48fa-a876-1047d6e987b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:49.163004 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86777a3f-458d-48fa-a876-1047d6e987b8 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:49.163284 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86777a3f-458d-48fa-a876-1047d6e987b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:49.163557 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86777a3f-458d-48fa-a876-1047d6e987b8 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:49.164501 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86777a3f-458d-48fa-a876-1047d6e987b8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:49.164875 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86777a3f-458d-48fa-a876-1047d6e987b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:49.166912 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86777a3f-458d-48fa-a876-1047d6e987b8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:49.202203 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 260/519] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Jun 6 10:24:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:49.203009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86777a3f-458d-48fa-a876-1047d6e987b8 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:49.223626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b270f-7610-40f5-a38f-f72667d89fe2 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:49.257555 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b5b14-3b08-4bfd-a400-99bcb26b472c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:49.258207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b5b14-3b08-4bfd-a400-99bcb26b472c None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:49.258487 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b5b14-3b08-4bfd-a400-99bcb26b472c None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:49.258761 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b5b14-3b08-4bfd-a400-99bcb26b472c None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:49.259475 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b5b14-3b08-4bfd-a400-99bcb26b472c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:49.259799 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b5b14-3b08-4bfd-a400-99bcb26b472c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:49.261186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b5b14-3b08-4bfd-a400-99bcb26b472c None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:49.274793 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 260/520] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:49.275575 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b5b14-3b08-4bfd-a400-99bcb26b472c None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:49.302147 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86777a3f-458d-48fa-a876-1047d6e987b8 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:49.329876 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:49.336663 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792054b-2322-4c26-bca2-5b3e262f67ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:49.337313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792054b-2322-4c26-bca2-5b3e262f67ce None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:49.337594 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792054b-2322-4c26-bca2-5b3e262f67ce None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:49.337871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792054b-2322-4c26-bca2-5b3e262f67ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:49.338694 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792054b-2322-4c26-bca2-5b3e262f67ce None admin] RBAC: Adding query filter params (role.id=95eccd30e39b4105ac6e8c0d9f973951, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=ed8a40e4f1034a8e99d0eb505e448db9) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:49.339002 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792054b-2322-4c26-bca2-5b3e262f67ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:49.340462 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792054b-2322-4c26-bca2-5b3e262f67ce None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:49.344665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:49.345780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 261/521] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:24:49 2022] GET /identity/v3/role_assignments?user.id=ed8a40e4f1034a8e99d0eb505e448db9&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=95eccd30e39b4105ac6e8c0d9f973951 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:49.346523 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792054b-2322-4c26-bca2-5b3e262f67ce None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:49.379831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b5b14-3b08-4bfd-a400-99bcb26b472c None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:49.423057 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88592687-4426-48e0-99ed-860995dbbc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:49.423921 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88592687-4426-48e0-99ed-860995dbbc37 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:49.424202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88592687-4426-48e0-99ed-860995dbbc37 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:49.424575 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88592687-4426-48e0-99ed-860995dbbc37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:49.436607 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88592687-4426-48e0-99ed-860995dbbc37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:49.438186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88592687-4426-48e0-99ed-860995dbbc37 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:49.458953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-88592687-4426-48e0-99ed-860995dbbc37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:49.458953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:49.458953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:49.458953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:49.458953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:49.458953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:49.458953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:49.458953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:49.458953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:49.458953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:49.458953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:49.458953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:49.458953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:49.458953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:49.458953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:49.459899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:49.459899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:49.459899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:49.459899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:49.459899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:49.459899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:49.459899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:49.459899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:49.459899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:49.459899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:49.459899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:49.459899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:49.459899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:49.459899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:49.459899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:49.459899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:49.459899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:49.460934 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:49.460934 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:49.460934 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:49.460934 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:49.460934 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 261/522] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:49.460934 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88592687-4426-48e0-99ed-860995dbbc37 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:49.464114 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792054b-2322-4c26-bca2-5b3e262f67ce None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:49.504482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dc413-1643-4294-8834-b3ed2acbea80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:49.505330 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dc413-1643-4294-8834-b3ed2acbea80 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:49.505609 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dc413-1643-4294-8834-b3ed2acbea80 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:49.505883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dc413-1643-4294-8834-b3ed2acbea80 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:49.506606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550dc413-1643-4294-8834-b3ed2acbea80 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:49.508106 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550dc413-1643-4294-8834-b3ed2acbea80 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:49.539817 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 262/523] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jun 6 10:24:49 2022] POST /identity/v3/users => generated 247 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:24:49.539817 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dc413-1643-4294-8834-b3ed2acbea80 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:49.596805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88592687-4426-48e0-99ed-860995dbbc37 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:49.609058 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:49.629295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:49.642670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd5ec8-9201-42ef-b651-61576c2687b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:49.643488 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd5ec8-9201-42ef-b651-61576c2687b8 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:49.643942 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd5ec8-9201-42ef-b651-61576c2687b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:49.644851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd5ec8-9201-42ef-b651-61576c2687b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:49.685020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dc413-1643-4294-8834-b3ed2acbea80 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:49.727058 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1ccf3-850f-4533-ab01-967843b1333e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:49.728273 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1ccf3-850f-4533-ab01-967843b1333e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:49.728769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1ccf3-850f-4533-ab01-967843b1333e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:49.729332 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1ccf3-850f-4533-ab01-967843b1333e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:49.731417 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd5ec8-9201-42ef-b651-61576c2687b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:49.733324 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd5ec8-9201-42ef-b651-61576c2687b8 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:49.738570 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1ccf3-850f-4533-ab01-967843b1333e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:49.740531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1ccf3-850f-4533-ab01-967843b1333e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:49.757111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-20e1ccf3-850f-4533-ab01-967843b1333e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 10:24:49.757111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:49.757111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:49.757111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:49.757111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:49.757111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:49.757111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:49.757111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:49.757111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:49.757111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:49.757111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:49.757111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:49.757111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:24:49.757111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:24:49.757111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:24:49.757111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:24:49.758116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:49.758116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:49.758116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:24:49.758116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:24:49.758116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:24:49.758116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:24:49.758116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:49.758116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:49.758116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:49.758116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:49.758116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:49.758116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:49.758116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:49.758116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:49.758116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:49.758116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:49.758116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:49.758116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:49.759024 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:49.759024 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:49.759024 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:24:49.759024 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:24:49.759024 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:24:49.759024 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:24:49.759024 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:24:49.759024 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:24:49.759024 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 10:24:49.759024 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:49.760966 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 263/524] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:24:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:49.762015 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1ccf3-850f-4533-ab01-967843b1333e None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:49.852378 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 262/525] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:49.853229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd5ec8-9201-42ef-b651-61576c2687b8 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:49.884405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:49.888705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1ccf3-850f-4533-ab01-967843b1333e None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:49.900151 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:49.934202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9bff70-b24a-4a0e-9674-d6c5b217baa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:49.935529 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bff70-b24a-4a0e-9674-d6c5b217baa7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:49.935955 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bff70-b24a-4a0e-9674-d6c5b217baa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:49.936337 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bff70-b24a-4a0e-9674-d6c5b217baa7 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:49.937465 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9bff70-b24a-4a0e-9674-d6c5b217baa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:49.937872 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9bff70-b24a-4a0e-9674-d6c5b217baa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:49.940319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9bff70-b24a-4a0e-9674-d6c5b217baa7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:49.960170 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 264/526] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:49.961298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9bff70-b24a-4a0e-9674-d6c5b217baa7 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:49.970555 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd5ec8-9201-42ef-b651-61576c2687b8 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:50.007938 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f1e00-59da-4628-a9d1-ccf28822411b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:50.008885 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f1e00-59da-4628-a9d1-ccf28822411b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:50.009335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f1e00-59da-4628-a9d1-ccf28822411b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:50.009762 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f1e00-59da-4628-a9d1-ccf28822411b None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:50.010746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f1e00-59da-4628-a9d1-ccf28822411b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:50.011202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f1e00-59da-4628-a9d1-ccf28822411b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:50.013296 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f1e00-59da-4628-a9d1-ccf28822411b None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:50.035666 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 263/527] 10.222.0.42 () {56 vars in 1139 bytes} [Mon Jun 6 10:24:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:50.037221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f1e00-59da-4628-a9d1-ccf28822411b None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:50.090793 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9bff70-b24a-4a0e-9674-d6c5b217baa7 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:50.127929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4531d57-1ebc-4d9e-a0eb-5bc1ecdbe5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:50.129624 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4531d57-1ebc-4d9e-a0eb-5bc1ecdbe5de None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:50.129906 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4531d57-1ebc-4d9e-a0eb-5bc1ecdbe5de None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:50.130270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4531d57-1ebc-4d9e-a0eb-5bc1ecdbe5de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:50.133165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:50.142609 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4531d57-1ebc-4d9e-a0eb-5bc1ecdbe5de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:50.145312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4531d57-1ebc-4d9e-a0eb-5bc1ecdbe5de None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:50.153340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f1e00-59da-4628-a9d1-ccf28822411b None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:50.166934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-a4531d57-1ebc-4d9e-a0eb-5bc1ecdbe5de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:50.166934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:50.166934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:50.166934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:50.166934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:50.166934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:50.166934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:50.166934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:50.166934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:50.166934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:50.166934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:50.166934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:50.166934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:50.166934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:50.166934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:50.168836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:50.168836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:50.168836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:50.168836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:50.168836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:50.168836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:50.168836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:50.168836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:50.168836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:50.168836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:50.168836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:50.168836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:50.168836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:50.168836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:50.168836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:50.168836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:50.168836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:50.169974 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:50.169974 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:50.169974 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:50.169974 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:50.171845 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 265/528] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:50.173386 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4531d57-1ebc-4d9e-a0eb-5bc1ecdbe5de None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:50.191964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45caf5-943e-4d79-892b-1130171383fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:50.192765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45caf5-943e-4d79-892b-1130171383fd None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:50.193058 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45caf5-943e-4d79-892b-1130171383fd None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:50.193410 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45caf5-943e-4d79-892b-1130171383fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:50.200615 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45caf5-943e-4d79-892b-1130171383fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:50.202075 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:50.203075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45caf5-943e-4d79-892b-1130171383fd None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:50.220582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-0c45caf5-943e-4d79-892b-1130171383fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:24:50.220582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:50.220582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:50.220582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:50.220582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:50.220582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:50.220582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:50.220582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:50.220582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:50.220582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:50.220582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:50.220582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:50.220582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:24:50.220582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:24:50.220582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:24:50.221498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:24:50.221498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:50.221498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:50.221498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:50.221498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:50.221498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:50.221498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:50.221498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:50.221498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:50.221498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:50.221498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:50.221498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:50.221498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:50.221498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:50.221498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:50.221498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:50.221498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:50.222483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:24:50.222483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:24:50.222483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:24:50.222483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:24:50.222483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:24:50.222483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:24:50.222483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:24:50.222483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:50.222483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 264/529] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:24:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:50.222483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45caf5-943e-4d79-892b-1130171383fd None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:50.288572 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4531d57-1ebc-4d9e-a0eb-5bc1ecdbe5de None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:50.325470 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ae51a-7eea-48a6-8545-7a1915624ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:50.326420 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ae51a-7eea-48a6-8545-7a1915624ebe None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:50.326901 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ae51a-7eea-48a6-8545-7a1915624ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:50.327309 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ae51a-7eea-48a6-8545-7a1915624ebe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:50.332357 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45caf5-943e-4d79-892b-1130171383fd None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:50.368962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510714b9-9cce-4efa-9867-f35bd8844aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:50.369765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510714b9-9cce-4efa-9867-f35bd8844aeb None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:50.370148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510714b9-9cce-4efa-9867-f35bd8844aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:50.370768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510714b9-9cce-4efa-9867-f35bd8844aeb None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:50.371479 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510714b9-9cce-4efa-9867-f35bd8844aeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:50.372016 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510714b9-9cce-4efa-9867-f35bd8844aeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:50.373590 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510714b9-9cce-4efa-9867-f35bd8844aeb None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:50.392735 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 265/530] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:50.393667 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510714b9-9cce-4efa-9867-f35bd8844aeb None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:50.397227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:50.408744 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ae51a-7eea-48a6-8545-7a1915624ebe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:50.412208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ae51a-7eea-48a6-8545-7a1915624ebe None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:50.437060 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:50.506504 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510714b9-9cce-4efa-9867-f35bd8844aeb None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:50.523184 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 266/531] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:50.524766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ae51a-7eea-48a6-8545-7a1915624ebe None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:50.549980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef0fcc-6a66-4511-82b5-8dfb39991c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:50.550611 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef0fcc-6a66-4511-82b5-8dfb39991c78 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:50.551384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef0fcc-6a66-4511-82b5-8dfb39991c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:50.551559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef0fcc-6a66-4511-82b5-8dfb39991c78 None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:50.552342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef0fcc-6a66-4511-82b5-8dfb39991c78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:50.552642 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef0fcc-6a66-4511-82b5-8dfb39991c78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:50.554182 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef0fcc-6a66-4511-82b5-8dfb39991c78 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:50.567254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 266/532] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:24:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:50.568186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef0fcc-6a66-4511-82b5-8dfb39991c78 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:50.634297 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ae51a-7eea-48a6-8545-7a1915624ebe None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:50.634988 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:50.663015 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:50.676741 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6448dca-7345-4b64-a37e-f1fc88f721f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:50.677902 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6448dca-7345-4b64-a37e-f1fc88f721f8 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:50.678408 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6448dca-7345-4b64-a37e-f1fc88f721f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:50.678950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6448dca-7345-4b64-a37e-f1fc88f721f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:50.680424 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef0fcc-6a66-4511-82b5-8dfb39991c78 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:50.686716 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6448dca-7345-4b64-a37e-f1fc88f721f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:50.688893 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6448dca-7345-4b64-a37e-f1fc88f721f8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:50.704924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-a6448dca-7345-4b64-a37e-f1fc88f721f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:24:50.704924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:50.704924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:50.704924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:50.704924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:50.704924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:50.704924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:50.704924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:50.704924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:50.704924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:50.704924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:50.704924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:50.704924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:24:50.704924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:24:50.704924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:24:50.705787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:24:50.705787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:50.705787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:50.705787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:50.705787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:50.705787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:50.705787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:50.705787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:50.705787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:50.705787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:50.705787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:50.705787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:50.705787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:50.705787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:50.705787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:50.705787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:50.705787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:50.706644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:24:50.706644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:24:50.706644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:24:50.706644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:24:50.706644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:24:50.706644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:24:50.706644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:24:50.706644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:50.709695 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 267/533] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:24:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:50.710666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6448dca-7345-4b64-a37e-f1fc88f721f8 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:50.717676 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17f5a1-d2f3-4bd9-8fe8-1609898f5831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:50.718473 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17f5a1-d2f3-4bd9-8fe8-1609898f5831 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:50.718755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17f5a1-d2f3-4bd9-8fe8-1609898f5831 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:50.719027 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17f5a1-d2f3-4bd9-8fe8-1609898f5831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:50.731175 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17f5a1-d2f3-4bd9-8fe8-1609898f5831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:50.732861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17f5a1-d2f3-4bd9-8fe8-1609898f5831 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:50.751874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-3a17f5a1-d2f3-4bd9-8fe8-1609898f5831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:50.751874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:50.751874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:50.751874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:50.751874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:50.751874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:50.751874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:50.751874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:50.751874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:50.751874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:50.751874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:50.751874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:50.751874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:50.751874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:50.751874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:50.752731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:50.752731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:50.752731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:50.752731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:50.752731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:50.752731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:50.752731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:50.752731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:50.752731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:50.752731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:50.752731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:50.752731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:50.752731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:50.752731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:50.752731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:50.752731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:50.752731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:50.753568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:50.753568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:50.753568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:50.753568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:50.757641 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 267/534] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:50.758538 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17f5a1-d2f3-4bd9-8fe8-1609898f5831 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:50.825302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6448dca-7345-4b64-a37e-f1fc88f721f8 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:50.862139 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309161b8-d798-4d05-9980-bd23903515dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:50.863816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309161b8-d798-4d05-9980-bd23903515dd None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:50.864393 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309161b8-d798-4d05-9980-bd23903515dd None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:50.864859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309161b8-d798-4d05-9980-bd23903515dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:50.866083 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309161b8-d798-4d05-9980-bd23903515dd None admin] RBAC: Adding query filter params (role.id=95eccd30e39b4105ac6e8c0d9f973951, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=2b8836193f1a47e1a62ea5167e01b26e) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:50.866589 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309161b8-d798-4d05-9980-bd23903515dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:50.868265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309161b8-d798-4d05-9980-bd23903515dd None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:50.868977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17f5a1-d2f3-4bd9-8fe8-1609898f5831 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:50.875039 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 268/535] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:24:50 2022] GET /identity/v3/role_assignments?user.id=2b8836193f1a47e1a62ea5167e01b26e&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=95eccd30e39b4105ac6e8c0d9f973951 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:50.876341 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309161b8-d798-4d05-9980-bd23903515dd None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:50.884774 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:50.907757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597be79-4f7b-480e-8d2d-e7325c6bdc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:50.908461 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597be79-4f7b-480e-8d2d-e7325c6bdc68 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:50.908743 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597be79-4f7b-480e-8d2d-e7325c6bdc68 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:50.909017 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597be79-4f7b-480e-8d2d-e7325c6bdc68 None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:50.910252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597be79-4f7b-480e-8d2d-e7325c6bdc68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:50.910950 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597be79-4f7b-480e-8d2d-e7325c6bdc68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:50.912312 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597be79-4f7b-480e-8d2d-e7325c6bdc68 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:50.927525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 268/536] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:24:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:50.934540 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597be79-4f7b-480e-8d2d-e7325c6bdc68 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:50.938196 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:51.000232 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309161b8-d798-4d05-9980-bd23903515dd None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:51.039765 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36a835-51bf-4518-b39a-55c328e4bb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:51.040668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36a835-51bf-4518-b39a-55c328e4bb2f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:51.041038 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36a835-51bf-4518-b39a-55c328e4bb2f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:51.041407 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36a835-51bf-4518-b39a-55c328e4bb2f None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:51.042329 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36a835-51bf-4518-b39a-55c328e4bb2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:51.042732 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36a835-51bf-4518-b39a-55c328e4bb2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:51.044404 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36a835-51bf-4518-b39a-55c328e4bb2f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:51.055231 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597be79-4f7b-480e-8d2d-e7325c6bdc68 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:51.059750 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 269/537] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:51.065038 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36a835-51bf-4518-b39a-55c328e4bb2f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:51.101472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083dc03-4462-4021-a48f-138bc38d2670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:51.102360 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083dc03-4462-4021-a48f-138bc38d2670 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:51.102802 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083dc03-4462-4021-a48f-138bc38d2670 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:51.103217 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083dc03-4462-4021-a48f-138bc38d2670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:51.104334 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083dc03-4462-4021-a48f-138bc38d2670 None admin] RBAC: Adding query filter params (role.id=95eccd30e39b4105ac6e8c0d9f973951, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=1a67e007fc314b34aac2471a7c20f240) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:51.104787 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083dc03-4462-4021-a48f-138bc38d2670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:51.106474 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083dc03-4462-4021-a48f-138bc38d2670 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:51.113897 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 269/538] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:24:50 2022] GET /identity/v3/role_assignments?user.id=1a67e007fc314b34aac2471a7c20f240&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=95eccd30e39b4105ac6e8c0d9f973951 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:51.184509 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36a835-51bf-4518-b39a-55c328e4bb2f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:51.193763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:51.226261 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522c51f-90f1-4437-bb0d-02d8f9881324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:51.227269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522c51f-90f1-4437-bb0d-02d8f9881324 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:51.227647 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522c51f-90f1-4437-bb0d-02d8f9881324 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:51.228069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522c51f-90f1-4437-bb0d-02d8f9881324 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:51.236181 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522c51f-90f1-4437-bb0d-02d8f9881324 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:51.237818 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522c51f-90f1-4437-bb0d-02d8f9881324 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:51.254996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-e522c51f-90f1-4437-bb0d-02d8f9881324 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:24:51.254996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:51.254996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:51.254996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:51.254996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:51.254996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:51.254996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:51.254996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:51.254996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:51.254996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:51.254996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:51.254996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:51.254996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:24:51.254996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:24:51.254996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:24:51.256104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:24:51.256104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:51.256104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:51.256104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:51.256104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:51.256104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:51.256104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:51.256104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:51.256104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:51.256104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:51.256104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:51.256104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:51.256104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:51.256104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:51.256104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:51.256104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:51.256104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:51.257137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:24:51.257137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:24:51.257137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:24:51.257137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:24:51.257137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:24:51.257137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:24:51.257137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:24:51.257137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:51.258984 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 270/539] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:24:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:51.264600 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083dc03-4462-4021-a48f-138bc38d2670 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:51.326490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:51.415381 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083dc03-4462-4021-a48f-138bc38d2670 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:51.465819 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28833125-3fda-461c-8e66-b7664ed1eebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:51.466487 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28833125-3fda-461c-8e66-b7664ed1eebf None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:51.466771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28833125-3fda-461c-8e66-b7664ed1eebf None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:51.467050 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28833125-3fda-461c-8e66-b7664ed1eebf None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:51.467865 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28833125-3fda-461c-8e66-b7664ed1eebf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:51.468178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28833125-3fda-461c-8e66-b7664ed1eebf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:51.469617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28833125-3fda-461c-8e66-b7664ed1eebf None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:51.487817 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 270/540] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:24:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:51.494726 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522c51f-90f1-4437-bb0d-02d8f9881324 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:51.628782 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522c51f-90f1-4437-bb0d-02d8f9881324 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:51.675266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539ffa3-0cdd-434a-a3cd-5d9d364c1f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:51.676832 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539ffa3-0cdd-434a-a3cd-5d9d364c1f40 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:51.677669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539ffa3-0cdd-434a-a3cd-5d9d364c1f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:51.678507 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539ffa3-0cdd-434a-a3cd-5d9d364c1f40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:51.680361 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539ffa3-0cdd-434a-a3cd-5d9d364c1f40 None admin] RBAC: Adding query filter params (role.id=95eccd30e39b4105ac6e8c0d9f973951, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=881069f9b40a485dac2dda684dd6cd3b) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:51.681153 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539ffa3-0cdd-434a-a3cd-5d9d364c1f40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:51.684009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539ffa3-0cdd-434a-a3cd-5d9d364c1f40 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:51.697832 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 271/541] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:24:51 2022] GET /identity/v3/role_assignments?user.id=881069f9b40a485dac2dda684dd6cd3b&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=95eccd30e39b4105ac6e8c0d9f973951 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:51.760194 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80261d-c997-475d-9348-3096470d60e3 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:51.760947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80261d-c997-475d-9348-3096470d60e3 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:51.761203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80261d-c997-475d-9348-3096470d60e3 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:51.763141 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 271/542] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:51.770027 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee43ed-1ce0-4b1f-8ebb-d193e8bd0573 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:51.770332 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee43ed-1ce0-4b1f-8ebb-d193e8bd0573 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:51.770617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee43ed-1ce0-4b1f-8ebb-d193e8bd0573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:51.790662 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-6eee43ed-1ce0-4b1f-8ebb-d193e8bd0573 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:51.791738 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6eee43ed-1ce0-4b1f-8ebb-d193e8bd0573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:51.820657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:51.895305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 272/543] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:51.941900 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80261d-c997-475d-9348-3096470d60e3 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:52.097517 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:52.103018 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80261d-c997-475d-9348-3096470d60e3 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:52.151711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa72651-35bc-471c-8d1a-acec8f877d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:52.152552 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa72651-35bc-471c-8d1a-acec8f877d60 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:52.152837 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa72651-35bc-471c-8d1a-acec8f877d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:52.153113 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa72651-35bc-471c-8d1a-acec8f877d60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:52.168797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa72651-35bc-471c-8d1a-acec8f877d60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:52.177211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-daa72651-35bc-471c-8d1a-acec8f877d60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:52.177211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:52.177211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:52.177211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:52.177211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:52.177211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:52.177211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:52.177211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:52.177211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:52.177211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:52.177211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:52.177211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:52.177211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:52.177211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:52.177211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:52.177211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:52.178269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:52.178269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:52.178269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:52.178269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:52.178269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:52.178269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:52.178269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:52.178269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:52.178269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:52.178269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:52.178269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:52.178269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:52.178269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:52.178269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:52.178269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:52.178269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:52.179087 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:52.179087 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:52.179087 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:52.179087 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:52.179087 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:52.179087 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:52.179087 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:52.179087 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:52.179087 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:52.179087 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:52.179087 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:52.179087 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:52.179087 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 272/544] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:24:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:52.184655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee43ed-1ce0-4b1f-8ebb-d193e8bd0573 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:52.320222 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d8fbc-b32a-416e-a661-ba357eee757e None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:52.320451 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d8fbc-b32a-416e-a661-ba357eee757e None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:52.320701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d8fbc-b32a-416e-a661-ba357eee757e None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:52.323169 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 273/545] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:52.328536 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c720df-bdd3-40ed-9924-54ce3bff7c7f None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:52.328858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c720df-bdd3-40ed-9924-54ce3bff7c7f None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:52.329134 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c720df-bdd3-40ed-9924-54ce3bff7c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:52.360411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee43ed-1ce0-4b1f-8ebb-d193e8bd0573 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:52.372787 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-a1c720df-bdd3-40ed-9924-54ce3bff7c7f None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:52.374075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a1c720df-bdd3-40ed-9924-54ce3bff7c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:52.411886 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93407230-c589-4c40-b1c0-f8028ad65eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:52.412900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93407230-c589-4c40-b1c0-f8028ad65eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:52.413388 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93407230-c589-4c40-b1c0-f8028ad65eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:52.413886 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93407230-c589-4c40-b1c0-f8028ad65eb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:52.414670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93407230-c589-4c40-b1c0-f8028ad65eb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:52.414985 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93407230-c589-4c40-b1c0-f8028ad65eb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:52.416402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93407230-c589-4c40-b1c0-f8028ad65eb0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:52.441450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 273/546] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jun 6 10:24:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:52.449677 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93407230-c589-4c40-b1c0-f8028ad65eb0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:52.503301 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 274/547] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:52.524179 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:52.545225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c720df-bdd3-40ed-9924-54ce3bff7c7f None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:52.681315 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93407230-c589-4c40-b1c0-f8028ad65eb0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:52.722170 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca71476-bb7a-44f7-80ff-6e2deb35d7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:52.724443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca71476-bb7a-44f7-80ff-6e2deb35d7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:52.725094 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca71476-bb7a-44f7-80ff-6e2deb35d7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:52.725612 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca71476-bb7a-44f7-80ff-6e2deb35d7b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:52.740547 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca71476-bb7a-44f7-80ff-6e2deb35d7b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:52.742839 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca71476-bb7a-44f7-80ff-6e2deb35d7b2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:52.744180 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:52.766674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-3ca71476-bb7a-44f7-80ff-6e2deb35d7b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:52.766674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:52.766674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:52.766674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:52.766674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:52.766674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:52.766674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:52.766674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:52.766674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:52.766674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:52.766674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:52.766674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:52.766674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:52.766674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:52.766674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:52.767672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:52.767672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:52.767672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:52.767672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:52.767672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:52.767672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:52.767672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:52.767672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:52.767672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:52.767672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:52.767672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:52.767672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:52.767672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:52.767672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:52.767672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:52.767672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:52.767672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:52.768561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:52.768561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:52.768561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:52.768561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:52.775936 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 274/548] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:24:52.779734 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca71476-bb7a-44f7-80ff-6e2deb35d7b2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:52.786171 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c720df-bdd3-40ed-9924-54ce3bff7c7f None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:52.855064 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef416e4-0074-40ea-a558-1902919e7385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:52.856078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef416e4-0074-40ea-a558-1902919e7385 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:52.856360 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef416e4-0074-40ea-a558-1902919e7385 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:52.856631 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef416e4-0074-40ea-a558-1902919e7385 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:52.874271 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:52.885024 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef416e4-0074-40ea-a558-1902919e7385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:52.898071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-eef416e4-0074-40ea-a558-1902919e7385 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:52.898071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:52.898071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:52.898071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:52.898071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:52.898071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:52.898071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:52.898071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:52.898071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:52.898071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:52.898071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:52.898071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:52.898071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:52.898071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:52.898071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:52.898071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:52.899813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:52.899813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:52.899813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:52.899813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:52.899813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:52.899813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:52.899813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:52.899813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:52.899813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:52.899813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:52.899813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:52.899813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:52.899813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:52.899813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:52.899813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:52.899813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:52.900800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:52.900800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:52.900800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:52.900800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:52.900800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:52.900800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:52.900800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:52.900800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:52.900800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:52.900800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:52.900800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:52.900800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:52.903613 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 275/549] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:24:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:24:52.915928 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef416e4-0074-40ea-a558-1902919e7385 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:52.916294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca71476-bb7a-44f7-80ff-6e2deb35d7b2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:53.029368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef416e4-0074-40ea-a558-1902919e7385 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:53.045413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9a41e-2688-46a1-882c-cfa819384b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:53.046100 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9a41e-2688-46a1-882c-cfa819384b39 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:53.046387 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9a41e-2688-46a1-882c-cfa819384b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:53.046666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9a41e-2688-46a1-882c-cfa819384b39 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:53.047398 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9a41e-2688-46a1-882c-cfa819384b39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:53.047703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9a41e-2688-46a1-882c-cfa819384b39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:53.049114 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9a41e-2688-46a1-882c-cfa819384b39 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:53.071275 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd39386-9d35-43bd-928e-cd9755047ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:53.072291 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd39386-9d35-43bd-928e-cd9755047ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:53.073679 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd39386-9d35-43bd-928e-cd9755047ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:53.075637 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd39386-9d35-43bd-928e-cd9755047ac7 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:53.076675 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd39386-9d35-43bd-928e-cd9755047ac7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:53.077190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd39386-9d35-43bd-928e-cd9755047ac7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:53.078098 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 275/550] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:53.079960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd39386-9d35-43bd-928e-cd9755047ac7 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:53.088219 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9a41e-2688-46a1-882c-cfa819384b39 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:53.102854 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 276/551] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jun 6 10:24:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:53.107941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd39386-9d35-43bd-928e-cd9755047ac7 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:53.288510 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:53.335267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9a41e-2688-46a1-882c-cfa819384b39 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:53.371515 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b4ae5-d739-4211-801e-e8e2bae43651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:53.372351 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b4ae5-d739-4211-801e-e8e2bae43651 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:53.372634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b4ae5-d739-4211-801e-e8e2bae43651 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:53.372906 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b4ae5-d739-4211-801e-e8e2bae43651 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:53.376129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:53.383669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b4ae5-d739-4211-801e-e8e2bae43651 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:53.385558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b4ae5-d739-4211-801e-e8e2bae43651 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:53.402702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-4a0b4ae5-d739-4211-801e-e8e2bae43651 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 10:24:53.402702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:53.402702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:53.402702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:53.402702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:53.402702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:53.402702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:53.402702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:53.402702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:53.402702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:53.402702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:53.402702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:53.402702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:24:53.402702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:24:53.402702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:24:53.402702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:24:53.403857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:53.403857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:53.403857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:24:53.403857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:24:53.403857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:24:53.403857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:24:53.403857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:53.403857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:53.403857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:53.403857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:53.403857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:53.403857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:53.403857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:53.403857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:53.403857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:53.403857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:53.403857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:53.403857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:53.404764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:53.404764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:53.404764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:24:53.404764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:24:53.404764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:24:53.404764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:24:53.404764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:24:53.404764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:24:53.404764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 10:24:53.404764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:53.404764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 276/552] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jun 6 10:24:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:24:53.409324 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b4ae5-d739-4211-801e-e8e2bae43651 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:53.453346 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd39386-9d35-43bd-928e-cd9755047ac7 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:53.517562 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8fa7e-cae0-4a0f-a349-3aea3c35f9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:53.518390 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8fa7e-cae0-4a0f-a349-3aea3c35f9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:53.518674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8fa7e-cae0-4a0f-a349-3aea3c35f9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:53.518952 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8fa7e-cae0-4a0f-a349-3aea3c35f9a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:53.520955 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b4ae5-d739-4211-801e-e8e2bae43651 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:53.541288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8fa7e-cae0-4a0f-a349-3aea3c35f9a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:53.542744 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8fa7e-cae0-4a0f-a349-3aea3c35f9a0 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:53.560512 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da91773-3aea-4b2a-9343-bc92a49487a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:53.561182 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da91773-3aea-4b2a-9343-bc92a49487a9 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:53.561466 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da91773-3aea-4b2a-9343-bc92a49487a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:53.561742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da91773-3aea-4b2a-9343-bc92a49487a9 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:53.562580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da91773-3aea-4b2a-9343-bc92a49487a9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:53.562888 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da91773-3aea-4b2a-9343-bc92a49487a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:53.564359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da91773-3aea-4b2a-9343-bc92a49487a9 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:53.572565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-6fe8fa7e-cae0-4a0f-a349-3aea3c35f9a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:53.572565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:53.572565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:53.572565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:53.572565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:53.572565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:53.572565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:53.572565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:53.572565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:53.572565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:53.572565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:53.572565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:53.572565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:53.572565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:53.572565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:53.574674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:53.574674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:53.574674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:53.574674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:53.574674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:53.574674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:53.574674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:53.574674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:53.574674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:53.574674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:53.574674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:53.574674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:53.574674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:53.574674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:53.574674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:53.574674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:53.574674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:53.576697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:53.576697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:53.576697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:53.576697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:53.580298 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 277/553] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 473 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:24:53.582925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 277/554] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jun 6 10:24:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:53.585819 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8fa7e-cae0-4a0f-a349-3aea3c35f9a0 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:53.589440 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da91773-3aea-4b2a-9343-bc92a49487a9 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:53.697074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:53.725855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:53.801531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da91773-3aea-4b2a-9343-bc92a49487a9 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:53.815462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8fa7e-cae0-4a0f-a349-3aea3c35f9a0 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:53.852896 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721a330-884e-4252-a3da-9136551b41bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:53.853563 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721a330-884e-4252-a3da-9136551b41bc None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:53.853848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721a330-884e-4252-a3da-9136551b41bc None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:53.854123 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721a330-884e-4252-a3da-9136551b41bc None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:53.854847 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721a330-884e-4252-a3da-9136551b41bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:53.855207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721a330-884e-4252-a3da-9136551b41bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:53.856948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721a330-884e-4252-a3da-9136551b41bc None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:53.870702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 278/555] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:53.875812 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cc8db-f9d1-4836-b31a-e00ca9cc1cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:53.876631 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cc8db-f9d1-4836-b31a-e00ca9cc1cec None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:53.876913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cc8db-f9d1-4836-b31a-e00ca9cc1cec None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:53.877188 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cc8db-f9d1-4836-b31a-e00ca9cc1cec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:53.879250 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721a330-884e-4252-a3da-9136551b41bc None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:53.896491 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cc8db-f9d1-4836-b31a-e00ca9cc1cec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:53.897969 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cc8db-f9d1-4836-b31a-e00ca9cc1cec None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:53.916117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-bf8cc8db-f9d1-4836-b31a-e00ca9cc1cec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:53.916117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:53.916117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:53.916117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:53.916117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:53.916117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:53.916117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:53.916117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:53.916117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:53.916117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:53.916117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:53.916117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:53.916117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:53.916117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:53.916117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:53.917090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:53.917090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:53.917090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:53.917090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:53.917090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:53.917090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:53.917090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:53.917090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:53.917090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:53.917090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:53.917090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:53.917090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:53.917090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:53.917090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:53.917090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:53.917090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:53.917090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:53.918385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:53.918385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:53.918385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:53.918385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:53.918385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 278/556] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:53.919063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02194442-0618-4755-ae54-f2b7130513f4 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:53.919227 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02194442-0618-4755-ae54-f2b7130513f4 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:53.923436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02194442-0618-4755-ae54-f2b7130513f4 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:53.923436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 279/557] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:53.923436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02194442-0618-4755-ae54-f2b7130513f4 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:53.997458 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721a330-884e-4252-a3da-9136551b41bc None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:54.021609 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:54.035740 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34270d96-dbd2-493b-9564-22e67eb2fcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:54.036566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34270d96-dbd2-493b-9564-22e67eb2fcd0 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:54.036838 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34270d96-dbd2-493b-9564-22e67eb2fcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:54.037108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34270d96-dbd2-493b-9564-22e67eb2fcd0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:54.037670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02194442-0618-4755-ae54-f2b7130513f4 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:54.040047 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:54.045573 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34270d96-dbd2-493b-9564-22e67eb2fcd0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:54.047008 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34270d96-dbd2-493b-9564-22e67eb2fcd0 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:54.063216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-34270d96-dbd2-493b-9564-22e67eb2fcd0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 10:24:54.063216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:54.063216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:54.063216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:54.063216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:54.063216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:54.063216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:54.063216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:54.063216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:54.063216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:54.063216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:54.063216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:54.063216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:24:54.063216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:24:54.063216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:24:54.063216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:24:54.064237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:54.064237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:54.064237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:24:54.064237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:24:54.064237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:24:54.064237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:24:54.064237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:54.064237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:54.064237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:54.064237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:54.064237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:54.064237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:54.064237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:54.064237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:54.064237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:54.064237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:54.064237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:54.064237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:54.065161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:54.065161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:54.065161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:24:54.065161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:24:54.065161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:24:54.065161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:24:54.065161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:24:54.065161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:24:54.065161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 10:24:54.065161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:54.065788 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 279/558] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:24:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:24:54.068052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c2343-9c87-4eac-adb3-1d2b31dd43f9 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:54.068499 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c2343-9c87-4eac-adb3-1d2b31dd43f9 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:54.068915 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c2343-9c87-4eac-adb3-1d2b31dd43f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:54.086117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c379ded-f84c-4fe1-a87d-e7c9982f27b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:54.087094 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c379ded-f84c-4fe1-a87d-e7c9982f27b1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:54.087216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-3a0c2343-9c87-4eac-adb3-1d2b31dd43f9 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:54.087302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c379ded-f84c-4fe1-a87d-e7c9982f27b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:54.087302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c379ded-f84c-4fe1-a87d-e7c9982f27b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:54.088073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c379ded-f84c-4fe1-a87d-e7c9982f27b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:54.088263 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-3a0c2343-9c87-4eac-adb3-1d2b31dd43f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:54.088369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c379ded-f84c-4fe1-a87d-e7c9982f27b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:54.090745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c379ded-f84c-4fe1-a87d-e7c9982f27b1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:54.103717 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 280/559] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:54.104905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdee436-9ccf-436a-bc09-f4fa0d38149b None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:54.105193 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdee436-9ccf-436a-bc09-f4fa0d38149b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:54.105469 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdee436-9ccf-436a-bc09-f4fa0d38149b None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:54.110749 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 281/560] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:54.110749 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdee436-9ccf-436a-bc09-f4fa0d38149b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:54.169917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 280/561] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:54.171023 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c2343-9c87-4eac-adb3-1d2b31dd43f9 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:54.233618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdee436-9ccf-436a-bc09-f4fa0d38149b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:54.271172 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb6c0d4-7c41-4db9-8531-0e16ce048e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:54.271857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6c0d4-7c41-4db9-8531-0e16ce048e0e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:54.272139 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6c0d4-7c41-4db9-8531-0e16ce048e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:54.272412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6c0d4-7c41-4db9-8531-0e16ce048e0e None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:54.273238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb6c0d4-7c41-4db9-8531-0e16ce048e0e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:54.273789 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb6c0d4-7c41-4db9-8531-0e16ce048e0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:54.275209 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb6c0d4-7c41-4db9-8531-0e16ce048e0e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:54.292560 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c2343-9c87-4eac-adb3-1d2b31dd43f9 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:54.293467 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 282/562] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Jun 6 10:24:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:54.299135 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4dc6b-3882-48a7-849e-73cee1c604ec None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:54.299430 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4dc6b-3882-48a7-849e-73cee1c604ec None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:54.299703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4dc6b-3882-48a7-849e-73cee1c604ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:54.308338 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:54.316774 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-8ca4dc6b-3882-48a7-849e-73cee1c604ec None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:54.317905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-8ca4dc6b-3882-48a7-849e-73cee1c604ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:54.333227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c80b72-9289-4ecf-9490-44e7f79d04e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:54.334015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c80b72-9289-4ecf-9490-44e7f79d04e3 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:54.334297 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c80b72-9289-4ecf-9490-44e7f79d04e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:54.334566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c80b72-9289-4ecf-9490-44e7f79d04e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:54.335243 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:54.341701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c80b72-9289-4ecf-9490-44e7f79d04e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:54.343124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c80b72-9289-4ecf-9490-44e7f79d04e3 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:54.357739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-68c80b72-9289-4ecf-9490-44e7f79d04e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:24:54.357739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:54.357739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:54.357739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:54.357739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:54.357739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:54.357739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:54.357739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:54.357739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:54.357739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:54.357739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:54.357739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:54.357739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:24:54.357739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:24:54.357739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:24:54.358703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:24:54.358703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:54.358703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:54.358703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:54.358703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:54.358703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:54.358703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:54.358703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:54.358703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:54.358703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:54.358703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:54.358703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:54.358703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:54.358703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:54.358703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:54.358703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:54.358703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:54.359605 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:24:54.359605 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:24:54.359605 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:24:54.359605 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:24:54.359605 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:24:54.359605 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:24:54.359605 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:24:54.359605 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:54.359605 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 281/563] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:24:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:54.360120 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c80b72-9289-4ecf-9490-44e7f79d04e3 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:54.403833 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 283/564] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:54.403833 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4dc6b-3882-48a7-849e-73cee1c604ec None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:54.492216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c80b72-9289-4ecf-9490-44e7f79d04e3 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:54.514216 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4dc6b-3882-48a7-849e-73cee1c604ec None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:54.525466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76d60b-076f-4979-bcb3-760912b2d889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:54.526258 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76d60b-076f-4979-bcb3-760912b2d889 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:54.526525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76d60b-076f-4979-bcb3-760912b2d889 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:54.526796 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76d60b-076f-4979-bcb3-760912b2d889 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:54.538626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76d60b-076f-4979-bcb3-760912b2d889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:54.546294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-6b76d60b-076f-4979-bcb3-760912b2d889 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:54.546294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:54.546294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:54.546294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:54.546294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:54.546294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:54.546294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:54.546294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:54.546294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:54.546294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:54.546294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:54.546294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:54.546294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:54.546294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:54.546294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:54.546294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:54.547228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:54.547228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:54.547228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:54.547228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:54.547228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:54.547228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:54.547228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:54.547228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:54.547228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:54.547228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:54.547228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:54.547228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:54.547228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:54.547228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:54.547228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:54.547228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:54.548480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:54.548480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:54.548480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:54.548480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:54.548480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:54.548480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:54.548480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:54.548480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:54.548480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:54.548480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:54.548480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:54.548480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:54.548480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 282/565] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:24:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:54.548480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76d60b-076f-4979-bcb3-760912b2d889 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:54.556721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5c2a7-b7d5-489f-880d-6817d6375207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:54.557380 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5c2a7-b7d5-489f-880d-6817d6375207 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:54.557661 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5c2a7-b7d5-489f-880d-6817d6375207 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:54.557931 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5c2a7-b7d5-489f-880d-6817d6375207 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:54.580136 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:54.583638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:54.632851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5c2a7-b7d5-489f-880d-6817d6375207 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:54.634309 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5c2a7-b7d5-489f-880d-6817d6375207 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:54.668203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76d60b-076f-4979-bcb3-760912b2d889 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:54.705244 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8114c-3fba-4684-a5e4-c226bbdc402a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:54.705900 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8114c-3fba-4684-a5e4-c226bbdc402a None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:54.706180 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8114c-3fba-4684-a5e4-c226bbdc402a None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:54.706451 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8114c-3fba-4684-a5e4-c226bbdc402a None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:54.708085 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8114c-3fba-4684-a5e4-c226bbdc402a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:54.708400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8114c-3fba-4684-a5e4-c226bbdc402a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:54.709826 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8114c-3fba-4684-a5e4-c226bbdc402a None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:54.722169 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 283/566] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:24:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:54.723313 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8114c-3fba-4684-a5e4-c226bbdc402a None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:54.739151 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 284/567] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:24:54.740129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5c2a7-b7d5-489f-880d-6817d6375207 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:54.792006 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:54.819063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:54.828595 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8114c-3fba-4684-a5e4-c226bbdc402a None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:54.839339 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5c2a7-b7d5-489f-880d-6817d6375207 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:54.864160 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08881a-3111-4bb6-81b5-7c15949c17f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:54.864949 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08881a-3111-4bb6-81b5-7c15949c17f6 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:54.865229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08881a-3111-4bb6-81b5-7c15949c17f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:54.865500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08881a-3111-4bb6-81b5-7c15949c17f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:54.873502 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e7961-1743-4389-b097-9cc1c448fdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:54.874163 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e7961-1743-4389-b097-9cc1c448fdbf None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:54.874445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e7961-1743-4389-b097-9cc1c448fdbf None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:54.874722 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e7961-1743-4389-b097-9cc1c448fdbf None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:54.875404 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec08881a-3111-4bb6-81b5-7c15949c17f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:54.875516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e7961-1743-4389-b097-9cc1c448fdbf None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:54.875796 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e7961-1743-4389-b097-9cc1c448fdbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:54.877180 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e7961-1743-4389-b097-9cc1c448fdbf None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:54.883773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-ec08881a-3111-4bb6-81b5-7c15949c17f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:54.883773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:54.883773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:54.883773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:54.883773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:54.883773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:54.883773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:54.883773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:54.883773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:54.883773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:54.883773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:54.883773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:54.883773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:54.883773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:54.883773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:54.883773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:54.884722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:54.884722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:54.884722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:54.884722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:54.884722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:54.884722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:54.884722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:54.884722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:54.884722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:54.884722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:54.884722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:54.884722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:54.884722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:54.884722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:54.884722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:54.884722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:54.885558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:54.885558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:54.885558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:54.885558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:54.885558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:54.885558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:54.885558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:54.885558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:54.885558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:54.885558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:54.885558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:54.885558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:54.885558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 284/568] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:24:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:54.886714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08881a-3111-4bb6-81b5-7c15949c17f6 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:54.887248 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 285/569] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jun 6 10:24:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:54.888595 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa5ba0-a87c-469e-b81a-b48b2d23b440 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:54.888890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa5ba0-a87c-469e-b81a-b48b2d23b440 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:54.889173 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa5ba0-a87c-469e-b81a-b48b2d23b440 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:54.890961 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 286/570] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:54.891655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa5ba0-a87c-469e-b81a-b48b2d23b440 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:54.989132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08881a-3111-4bb6-81b5-7c15949c17f6 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:54.993301 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa5ba0-a87c-469e-b81a-b48b2d23b440 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:55.006414 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:55.021440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ebaeb-1179-4bbf-926b-0205d04f5b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:55.022313 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ebaeb-1179-4bbf-926b-0205d04f5b20 None admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:55.022597 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ebaeb-1179-4bbf-926b-0205d04f5b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:55.022865 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ebaeb-1179-4bbf-926b-0205d04f5b20 None admin] PATH_INFO: `/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/ed8a40e4f1034a8e99d0eb505e448db9/roles/95eccd30e39b4105ac6e8c0d9f973951` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:55.026653 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1407595-9506-4024-bb9b-fd3a347de8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:55.027432 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1407595-9506-4024-bb9b-fd3a347de8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:55.027720 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1407595-9506-4024-bb9b-fd3a347de8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:55.028091 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1407595-9506-4024-bb9b-fd3a347de8c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:55.037929 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ebaeb-1179-4bbf-926b-0205d04f5b20 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f082d7ed3694fd9998623f3d4a5fea9, user_id=ed8a40e4f1034a8e99d0eb505e448db9, role_id=95eccd30e39b4105ac6e8c0d9f973951)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:55.038834 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1407595-9506-4024-bb9b-fd3a347de8c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:55.039353 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ebaeb-1179-4bbf-926b-0205d04f5b20 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:55.040285 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1407595-9506-4024-bb9b-fd3a347de8c0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:55.052446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 285/571] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:24:54 2022] PUT /identity/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/ed8a40e4f1034a8e99d0eb505e448db9/roles/95eccd30e39b4105ac6e8c0d9f973951 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:24:55.053019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ebaeb-1179-4bbf-926b-0205d04f5b20 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:55.055488 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:55.057727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-f1407595-9506-4024-bb9b-fd3a347de8c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:55.057727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:55.057727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:55.057727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:55.057727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:55.057727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:55.057727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:55.057727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:55.057727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:55.057727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:55.057727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:55.057727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:55.057727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:55.057727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:55.057727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:55.058679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:55.058679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:55.058679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:55.058679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:55.058679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:55.058679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:55.058679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:55.058679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:55.058679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:55.058679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:55.058679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:55.058679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:55.058679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:55.058679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:55.058679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:55.058679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:55.058679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:55.060151 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:55.060151 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:55.060151 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:55.060151 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:55.063850 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 287/572] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:55.064844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1407595-9506-4024-bb9b-fd3a347de8c0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:55.174090 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ebaeb-1179-4bbf-926b-0205d04f5b20 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:55.188092 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1407595-9506-4024-bb9b-fd3a347de8c0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:55.211646 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af34bb3-6627-4a33-af3f-860f570a8e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:55.212353 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af34bb3-6627-4a33-af3f-860f570a8e85 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:55.212637 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af34bb3-6627-4a33-af3f-860f570a8e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:55.212914 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af34bb3-6627-4a33-af3f-860f570a8e85 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:55.213665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af34bb3-6627-4a33-af3f-860f570a8e85 None admin] RBAC: Adding query filter params (name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:55.213971 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af34bb3-6627-4a33-af3f-860f570a8e85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:55.215355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af34bb3-6627-4a33-af3f-860f570a8e85 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:55.224801 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 286/573] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jun 6 10:24:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:55.225243 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eca072-0961-4cbb-834f-6acc532c9c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:55.226096 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6318090-83d1-4b03-a7cf-a0283486f021 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:55.226351 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eca072-0961-4cbb-834f-6acc532c9c18 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:55.226449 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6318090-83d1-4b03-a7cf-a0283486f021 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:55.226637 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6318090-83d1-4b03-a7cf-a0283486f021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:55.226974 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eca072-0961-4cbb-834f-6acc532c9c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:55.227433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eca072-0961-4cbb-834f-6acc532c9c18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:55.242252 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eca072-0961-4cbb-834f-6acc532c9c18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:55.243986 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eca072-0961-4cbb-834f-6acc532c9c18 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:55.246349 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-e6318090-83d1-4b03-a7cf-a0283486f021 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:55.247546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-e6318090-83d1-4b03-a7cf-a0283486f021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:55.267496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-c7eca072-0961-4cbb-834f-6acc532c9c18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:55.267496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:55.267496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:55.267496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:55.267496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:55.267496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:55.267496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:55.267496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:55.267496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:55.267496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:55.267496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:55.267496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:55.267496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:55.267496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:55.267496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:55.268444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:55.268444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:55.268444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:55.268444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:55.268444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:55.268444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:55.268444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:55.268444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:55.268444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:55.268444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:55.268444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:55.268444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:55.268444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:55.268444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:55.268444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:55.268444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:55.268444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:55.269429 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:55.269429 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:55.269429 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:55.269429 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:55.271365 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 288/574] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:55.272413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eca072-0961-4cbb-834f-6acc532c9c18 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:55.293790 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:55.315604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:55.324980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 287/575] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:55.325759 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6318090-83d1-4b03-a7cf-a0283486f021 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:55.397464 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eca072-0961-4cbb-834f-6acc532c9c18 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:55.433921 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58491230-72d2-45f3-b822-542589ac85f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:55.434677 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58491230-72d2-45f3-b822-542589ac85f1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:55.434961 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58491230-72d2-45f3-b822-542589ac85f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:55.435237 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58491230-72d2-45f3-b822-542589ac85f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:55.435983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58491230-72d2-45f3-b822-542589ac85f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:55.436291 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58491230-72d2-45f3-b822-542589ac85f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:55.437651 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58491230-72d2-45f3-b822-542589ac85f1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:55.447642 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6318090-83d1-4b03-a7cf-a0283486f021 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:55.449777 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 289/576] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:55.451450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58491230-72d2-45f3-b822-542589ac85f1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:55.509000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65ad8a-924b-48c4-9a05-148ce0536b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:55.510118 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65ad8a-924b-48c4-9a05-148ce0536b66 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:55.510695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65ad8a-924b-48c4-9a05-148ce0536b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:55.510971 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65ad8a-924b-48c4-9a05-148ce0536b66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:55.524567 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65ad8a-924b-48c4-9a05-148ce0536b66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:55.526036 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65ad8a-924b-48c4-9a05-148ce0536b66 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:55.544457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-ac65ad8a-924b-48c4-9a05-148ce0536b66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:55.544457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:55.544457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:55.544457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:55.544457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:55.544457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:55.544457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:55.544457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:55.544457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:55.544457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:55.544457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:55.544457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:55.544457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:55.544457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:55.544457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:55.545391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:55.545391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:55.545391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:55.545391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:55.545391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:55.545391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:55.545391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:55.545391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:55.545391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:55.545391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:55.545391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:55.545391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:55.545391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:55.545391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:55.545391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:55.545391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:55.545391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:55.546854 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:55.546854 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:55.546854 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:55.546854 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:55.546854 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 288/577] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:55.546854 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65ad8a-924b-48c4-9a05-148ce0536b66 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:55.556192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:55.586760 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58491230-72d2-45f3-b822-542589ac85f1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:55.596268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:55.626855 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232959dd-7f28-47ae-b1ac-54fb23cddee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:55.627604 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232959dd-7f28-47ae-b1ac-54fb23cddee1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:55.627916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232959dd-7f28-47ae-b1ac-54fb23cddee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:55.628188 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232959dd-7f28-47ae-b1ac-54fb23cddee1 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:55.629083 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232959dd-7f28-47ae-b1ac-54fb23cddee1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:55.629623 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232959dd-7f28-47ae-b1ac-54fb23cddee1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:55.630651 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232959dd-7f28-47ae-b1ac-54fb23cddee1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:55.642023 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 290/578] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:55.643054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232959dd-7f28-47ae-b1ac-54fb23cddee1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:55.660434 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65ad8a-924b-48c4-9a05-148ce0536b66 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:55.697396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b8cb0-bb0b-491b-86d3-bc1990a1cab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:55.698688 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b8cb0-bb0b-491b-86d3-bc1990a1cab7 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:55.699164 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b8cb0-bb0b-491b-86d3-bc1990a1cab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:55.699628 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b8cb0-bb0b-491b-86d3-bc1990a1cab7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:55.712943 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b8cb0-bb0b-491b-86d3-bc1990a1cab7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:55.720722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-da8b8cb0-bb0b-491b-86d3-bc1990a1cab7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:55.720722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:55.720722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:55.720722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:55.720722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:55.720722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:55.720722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:55.720722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:55.720722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:55.720722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:55.720722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:55.720722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:55.720722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:55.720722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:55.720722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:55.720722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:55.721638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:55.721638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:55.721638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:55.721638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:55.721638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:55.721638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:55.721638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:55.721638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:55.721638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:55.721638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:55.721638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:55.721638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:55.721638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:55.721638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:55.721638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:55.721638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:55.723028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:55.723028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:55.723028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:55.723028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:55.723028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:55.723028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:55.723028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:55.723028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:55.723028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:55.723028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:55.723028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:55.723028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:55.723028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 289/579] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:24:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:55.727829 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b8cb0-bb0b-491b-86d3-bc1990a1cab7 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:55.756701 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232959dd-7f28-47ae-b1ac-54fb23cddee1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:55.803276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bcfb6-c46b-48aa-b3f2-4a0368d543a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:55.804118 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bcfb6-c46b-48aa-b3f2-4a0368d543a2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:55.804399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bcfb6-c46b-48aa-b3f2-4a0368d543a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:55.804669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bcfb6-c46b-48aa-b3f2-4a0368d543a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:55.811435 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bcfb6-c46b-48aa-b3f2-4a0368d543a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:55.812922 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bcfb6-c46b-48aa-b3f2-4a0368d543a2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:55.814647 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:55.828991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-825bcfb6-c46b-48aa-b3f2-4a0368d543a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:24:55.828991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:55.828991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:55.828991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:55.828991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:55.828991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:55.828991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:55.828991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:55.828991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:55.828991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:55.828991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:55.828991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:55.828991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:24:55.828991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:24:55.828991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:24:55.829925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:24:55.829925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:55.829925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:55.829925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:55.829925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:55.829925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:55.829925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:55.829925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:55.829925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:55.829925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:55.829925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:55.829925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:55.829925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:55.829925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:55.829925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:55.829925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:55.829925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:55.830823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:24:55.830823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:24:55.830823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:24:55.830823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:24:55.830823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:24:55.830823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:24:55.830823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:24:55.830823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:55.835513 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 291/580] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:24:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:55.836518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bcfb6-c46b-48aa-b3f2-4a0368d543a2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:55.845714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b8cb0-bb0b-491b-86d3-bc1990a1cab7 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:55.855163 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:55.885635 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed42c0-b953-4f04-b9df-a5f8e3dfac20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:55.886295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed42c0-b953-4f04-b9df-a5f8e3dfac20 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:55.886595 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed42c0-b953-4f04-b9df-a5f8e3dfac20 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:55.886858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed42c0-b953-4f04-b9df-a5f8e3dfac20 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:55.887580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed42c0-b953-4f04-b9df-a5f8e3dfac20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:55.887938 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed42c0-b953-4f04-b9df-a5f8e3dfac20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:55.889306 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed42c0-b953-4f04-b9df-a5f8e3dfac20 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:55.902374 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 290/581] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:55.903172 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed42c0-b953-4f04-b9df-a5f8e3dfac20 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:55.950082 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bcfb6-c46b-48aa-b3f2-4a0368d543a2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:55.988654 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56a35c-87ec-456c-9c74-221269afcb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:55.989707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56a35c-87ec-456c-9c74-221269afcb38 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:55.990359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56a35c-87ec-456c-9c74-221269afcb38 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:55.990647 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56a35c-87ec-456c-9c74-221269afcb38 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:55.998035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56a35c-87ec-456c-9c74-221269afcb38 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:55.999481 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56a35c-87ec-456c-9c74-221269afcb38 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:56.010714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed42c0-b953-4f04-b9df-a5f8e3dfac20 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:56.014641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-4c56a35c-87ec-456c-9c74-221269afcb38 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 10:24:56.014641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:56.014641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:56.014641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:56.014641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:56.014641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:56.014641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:56.014641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:56.014641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:56.014641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:56.014641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:56.014641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:56.014641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:24:56.014641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:24:56.014641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:24:56.014641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:24:56.015764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:56.015764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:56.015764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:24:56.015764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:24:56.015764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:24:56.015764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:24:56.015764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:56.015764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:56.015764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:56.015764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:56.015764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:56.015764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:56.015764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:56.015764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:56.015764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:56.015764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:56.015764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:56.015764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:56.016807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:56.016807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:56.016807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:24:56.016807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:24:56.016807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:24:56.016807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:24:56.016807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:24:56.016807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:24:56.016807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 10:24:56.016807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:56.016807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 292/582] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:56.016807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56a35c-87ec-456c-9c74-221269afcb38 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:56.059534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2af716-f9cf-4794-a9fd-8ef72db00ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:56.060449 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2af716-f9cf-4794-a9fd-8ef72db00ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:56.060739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2af716-f9cf-4794-a9fd-8ef72db00ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:56.061097 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2af716-f9cf-4794-a9fd-8ef72db00ab9 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:56.061881 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2af716-f9cf-4794-a9fd-8ef72db00ab9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:56.062191 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2af716-f9cf-4794-a9fd-8ef72db00ab9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:56.063565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2af716-f9cf-4794-a9fd-8ef72db00ab9 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:56.073114 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 291/583] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jun 6 10:24:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:56.073719 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2af716-f9cf-4794-a9fd-8ef72db00ab9 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:56.084391 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:56.105918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:56.131139 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56a35c-87ec-456c-9c74-221269afcb38 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:56.167173 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b99662-2c54-440a-bd83-4a739fb862bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:56.167902 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b99662-2c54-440a-bd83-4a739fb862bb None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:56.168184 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b99662-2c54-440a-bd83-4a739fb862bb None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:56.168455 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b99662-2c54-440a-bd83-4a739fb862bb None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:56.169280 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b99662-2c54-440a-bd83-4a739fb862bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:56.169587 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b99662-2c54-440a-bd83-4a739fb862bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:56.171054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b99662-2c54-440a-bd83-4a739fb862bb None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:56.185284 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2af716-f9cf-4794-a9fd-8ef72db00ab9 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:56.195315 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 293/584] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:24:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:56.195315 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b99662-2c54-440a-bd83-4a739fb862bb None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:56.227121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739beb6d-ea31-448c-a964-8639a7b22b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:56.227855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739beb6d-ea31-448c-a964-8639a7b22b8f None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:56.228138 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739beb6d-ea31-448c-a964-8639a7b22b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:56.228532 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739beb6d-ea31-448c-a964-8639a7b22b8f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:56.236607 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739beb6d-ea31-448c-a964-8639a7b22b8f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:56.238791 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739beb6d-ea31-448c-a964-8639a7b22b8f None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:56.254920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-739beb6d-ea31-448c-a964-8639a7b22b8f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 06 10:24:56.254920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:56.254920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:56.254920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:56.254920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:56.254920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:56.254920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:56.254920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:56.254920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:56.254920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:56.254920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:56.254920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:56.254920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:24:56.254920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:24:56.254920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:24:56.254920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:24:56.256203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:56.256203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:56.256203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:24:56.256203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:24:56.256203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:24:56.256203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:24:56.256203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:56.256203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:56.256203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:56.256203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:56.256203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:56.256203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:56.256203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:56.256203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:56.256203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:56.256203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:56.256203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:56.256203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:56.257263 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:56.257263 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:56.257263 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:24:56.257263 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:24:56.257263 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:24:56.257263 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:24:56.257263 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:24:56.257263 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:24:56.257263 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 06 10:24:56.257263 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:56.257263 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 292/585] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:24:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:56.257880 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739beb6d-ea31-448c-a964-8639a7b22b8f None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:56.302958 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b99662-2c54-440a-bd83-4a739fb862bb None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:56.325454 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:56.350958 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842c32c-a115-4e69-843d-90e7baa764c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:56.351633 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842c32c-a115-4e69-843d-90e7baa764c7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:56.351945 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842c32c-a115-4e69-843d-90e7baa764c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:56.352223 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842c32c-a115-4e69-843d-90e7baa764c7 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:56.353059 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b842c32c-a115-4e69-843d-90e7baa764c7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:56.353366 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b842c32c-a115-4e69-843d-90e7baa764c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:56.354803 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b842c32c-a115-4e69-843d-90e7baa764c7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:56.356045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:56.369179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739beb6d-ea31-448c-a964-8639a7b22b8f None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:56.379149 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 294/586] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Jun 6 10:24:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:56.379149 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842c32c-a115-4e69-843d-90e7baa764c7 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:56.413433 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b593f-4dd9-4344-9e28-5bf8ff362a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:56.414246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b593f-4dd9-4344-9e28-5bf8ff362a9b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:56.414507 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b593f-4dd9-4344-9e28-5bf8ff362a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:56.414781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b593f-4dd9-4344-9e28-5bf8ff362a9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:56.425074 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b593f-4dd9-4344-9e28-5bf8ff362a9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:56.426535 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b593f-4dd9-4344-9e28-5bf8ff362a9b None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:56.444506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-273b593f-4dd9-4344-9e28-5bf8ff362a9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:56.444506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:56.444506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:56.444506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:56.444506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:56.444506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:56.444506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:56.444506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:56.444506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:56.444506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:56.444506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:56.444506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:56.444506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:56.444506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:56.444506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:56.445588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:56.445588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:56.445588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:56.445588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:56.445588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:56.445588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:56.445588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:56.445588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:56.445588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:56.445588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:56.445588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:56.445588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:56.445588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:56.445588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:56.445588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:56.445588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:56.445588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:56.446691 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:56.446691 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:56.446691 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:56.446691 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:56.447527 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 293/587] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:56.452475 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b593f-4dd9-4344-9e28-5bf8ff362a9b None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:56.489885 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842c32c-a115-4e69-843d-90e7baa764c7 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:56.526697 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d2449c-2731-4072-869f-fab92d431c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:56.527356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2449c-2731-4072-869f-fab92d431c8d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:56.527639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2449c-2731-4072-869f-fab92d431c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:56.527959 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2449c-2731-4072-869f-fab92d431c8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:56.528893 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d2449c-2731-4072-869f-fab92d431c8d None admin] RBAC: Adding query filter params (role.id=95eccd30e39b4105ac6e8c0d9f973951, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=fede9e52ad274285b09d748a684ede38) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:56.529210 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d2449c-2731-4072-869f-fab92d431c8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:56.530784 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d2449c-2731-4072-869f-fab92d431c8d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:56.537413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 295/588] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:24:56 2022] GET /identity/v3/role_assignments?user.id=fede9e52ad274285b09d748a684ede38&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=95eccd30e39b4105ac6e8c0d9f973951 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:56.538234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d2449c-2731-4072-869f-fab92d431c8d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:56.569011 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b593f-4dd9-4344-9e28-5bf8ff362a9b None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:56.587848 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:56.611718 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4ec95-e25f-4246-9213-43d9cf9ce2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:56.613463 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4ec95-e25f-4246-9213-43d9cf9ce2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:56.614015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4ec95-e25f-4246-9213-43d9cf9ce2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:56.614501 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4ec95-e25f-4246-9213-43d9cf9ce2a5 None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:56.615567 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab4ec95-e25f-4246-9213-43d9cf9ce2a5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:56.616146 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab4ec95-e25f-4246-9213-43d9cf9ce2a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:56.617969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab4ec95-e25f-4246-9213-43d9cf9ce2a5 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:56.629193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:56.639240 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 294/589] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Jun 6 10:24:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:56.640402 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4ec95-e25f-4246-9213-43d9cf9ce2a5 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:56.661921 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d2449c-2731-4072-869f-fab92d431c8d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:56.702189 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60133d-b562-4916-9201-8a627fa35c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:56.703320 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60133d-b562-4916-9201-8a627fa35c07 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:56.703906 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60133d-b562-4916-9201-8a627fa35c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:56.704345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60133d-b562-4916-9201-8a627fa35c07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:56.716945 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60133d-b562-4916-9201-8a627fa35c07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:56.718964 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60133d-b562-4916-9201-8a627fa35c07 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:56.741457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-ab60133d-b562-4916-9201-8a627fa35c07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:56.741457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:56.741457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:56.741457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:56.741457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:56.741457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:56.741457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:56.741457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:56.741457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:56.741457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:56.741457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:56.741457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:56.741457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:56.741457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:56.741457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:56.742576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:56.742576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:56.742576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:56.742576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:56.742576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:56.742576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:56.742576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:56.742576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:56.742576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:56.742576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:56.742576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:56.742576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:56.742576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:56.742576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:56.742576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:56.742576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:56.742576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:56.743461 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:56.743461 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:56.743461 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:56.743461 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:56.745491 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 296/590] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:56.746515 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60133d-b562-4916-9201-8a627fa35c07 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:56.759374 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4ec95-e25f-4246-9213-43d9cf9ce2a5 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:56.798240 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00f17c-e03a-4244-9758-27abc830dfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:56.798935 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00f17c-e03a-4244-9758-27abc830dfe1 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:56.799224 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00f17c-e03a-4244-9758-27abc830dfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:56.799501 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00f17c-e03a-4244-9758-27abc830dfe1 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:56.800958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00f17c-e03a-4244-9758-27abc830dfe1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:56.801265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00f17c-e03a-4244-9758-27abc830dfe1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:56.802832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00f17c-e03a-4244-9758-27abc830dfe1 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:56.819940 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 295/591] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:56.820978 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00f17c-e03a-4244-9758-27abc830dfe1 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:56.845403 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:56.855860 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60133d-b562-4916-9201-8a627fa35c07 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:56.887272 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:56.893315 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d9202-851e-40b9-a68a-2f93199faace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:56.894848 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d9202-851e-40b9-a68a-2f93199faace None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:56.895349 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d9202-851e-40b9-a68a-2f93199faace None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:56.895866 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d9202-851e-40b9-a68a-2f93199faace None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:56.906843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d9202-851e-40b9-a68a-2f93199faace None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:56.908518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d9202-851e-40b9-a68a-2f93199faace None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:56.928811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-1f8d9202-851e-40b9-a68a-2f93199faace None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:56.928811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:56.928811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:56.928811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:56.928811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:56.928811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:56.928811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:56.928811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:56.928811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:56.928811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:56.928811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:56.928811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:56.928811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:56.928811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:56.928811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:56.929954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:56.929954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:56.929954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:56.929954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:56.929954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:56.929954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:56.929954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:56.929954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:56.929954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:56.929954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:56.929954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:56.929954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:56.929954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:56.929954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:56.929954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:56.929954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:56.929954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:56.931034 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:56.931034 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:56.931034 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:56.931034 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:56.933025 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 297/592] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:56.933985 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d9202-851e-40b9-a68a-2f93199faace None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:56.937003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00f17c-e03a-4244-9758-27abc830dfe1 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:56.980390 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02205e08-d8ff-459b-9218-7edc4aecfc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:56.981051 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02205e08-d8ff-459b-9218-7edc4aecfc68 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:56.981331 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02205e08-d8ff-459b-9218-7edc4aecfc68 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:56.981602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02205e08-d8ff-459b-9218-7edc4aecfc68 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:56.982377 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02205e08-d8ff-459b-9218-7edc4aecfc68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:56.982682 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02205e08-d8ff-459b-9218-7edc4aecfc68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:56.984725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02205e08-d8ff-459b-9218-7edc4aecfc68 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:57.007645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 296/593] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:57.008639 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02205e08-d8ff-459b-9218-7edc4aecfc68 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:57.068363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d9202-851e-40b9-a68a-2f93199faace None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:57.106986 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd61ead-c298-46af-8549-c950958646eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:57.108190 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd61ead-c298-46af-8549-c950958646eb None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:57.108546 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd61ead-c298-46af-8549-c950958646eb None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:57.108896 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd61ead-c298-46af-8549-c950958646eb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:57.116552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd61ead-c298-46af-8549-c950958646eb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:57.118224 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd61ead-c298-46af-8549-c950958646eb None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:57.136600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-3cd61ead-c298-46af-8549-c950958646eb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 10:24:57.136600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:57.136600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:57.136600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:57.136600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:57.136600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:57.136600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:57.136600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:57.136600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:57.136600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:57.136600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:57.136600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:57.136600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:24:57.136600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:24:57.136600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:24:57.136600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:24:57.137769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:57.137769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:57.137769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:24:57.137769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:24:57.137769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:24:57.137769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:24:57.137769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:57.137769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:57.137769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:57.137769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:57.137769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:57.137769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:57.137769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:57.137769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:57.137769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:57.137769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:57.137769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:57.137769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:57.138938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:57.138938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:57.138938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:24:57.138938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:24:57.138938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:24:57.138938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:24:57.138938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:24:57.138938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:24:57.138938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 10:24:57.138938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:57.140662 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 298/594] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:24:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:57.141820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd61ead-c298-46af-8549-c950958646eb None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:57.147491 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:57.148777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02205e08-d8ff-459b-9218-7edc4aecfc68 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:57.174484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:57.193863 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7752d11-7f23-4a01-ab14-b6f0b1d3c857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:57.194523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7752d11-7f23-4a01-ab14-b6f0b1d3c857 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:57.194805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7752d11-7f23-4a01-ab14-b6f0b1d3c857 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:57.195079 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7752d11-7f23-4a01-ab14-b6f0b1d3c857 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:57.195896 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7752d11-7f23-4a01-ab14-b6f0b1d3c857 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:57.196206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7752d11-7f23-4a01-ab14-b6f0b1d3c857 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:57.197626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7752d11-7f23-4a01-ab14-b6f0b1d3c857 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:57.210756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 297/595] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:57.211602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7752d11-7f23-4a01-ab14-b6f0b1d3c857 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:57.284532 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd61ead-c298-46af-8549-c950958646eb None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:57.321008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce684ec-b4fa-453b-82c3-239960e69a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:57.321866 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce684ec-b4fa-453b-82c3-239960e69a37 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:57.322150 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce684ec-b4fa-453b-82c3-239960e69a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:57.322428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce684ec-b4fa-453b-82c3-239960e69a37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:57.329512 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce684ec-b4fa-453b-82c3-239960e69a37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:57.330935 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce684ec-b4fa-453b-82c3-239960e69a37 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:57.335009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7752d11-7f23-4a01-ab14-b6f0b1d3c857 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:57.346929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-fce684ec-b4fa-453b-82c3-239960e69a37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:24:57.346929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:57.346929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:57.346929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:57.346929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:57.346929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:57.346929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:57.346929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:57.346929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:57.346929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:57.346929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:57.346929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:57.346929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:24:57.346929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:24:57.346929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:24:57.348428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:24:57.348428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:57.348428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:57.348428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:57.348428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:57.348428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:57.348428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:57.348428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:57.348428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:57.348428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:57.348428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:57.348428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:57.348428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:57.348428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:57.348428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:57.348428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:57.348428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:57.349569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:24:57.349569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:24:57.349569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:24:57.349569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:24:57.349569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:24:57.349569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:24:57.349569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:24:57.349569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:57.349569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 299/596] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:24:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:57.349569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce684ec-b4fa-453b-82c3-239960e69a37 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:57.372233 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a40b82-a7ac-4a65-9066-f43bb0e20fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:57.372897 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a40b82-a7ac-4a65-9066-f43bb0e20fff None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:57.373177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a40b82-a7ac-4a65-9066-f43bb0e20fff None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:57.373450 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a40b82-a7ac-4a65-9066-f43bb0e20fff None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:57.374276 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a40b82-a7ac-4a65-9066-f43bb0e20fff None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:57.374584 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a40b82-a7ac-4a65-9066-f43bb0e20fff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:57.376028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a40b82-a7ac-4a65-9066-f43bb0e20fff None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:57.398230 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 298/597] 10.222.0.42 () {56 vars in 1139 bytes} [Mon Jun 6 10:24:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:57.400217 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ceb1d-fa25-4767-87c1-8e7866bd255f None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:57.400696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ceb1d-fa25-4767-87c1-8e7866bd255f None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:57.401146 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ceb1d-fa25-4767-87c1-8e7866bd255f None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:57.406530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 299/598] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:57.406530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ceb1d-fa25-4767-87c1-8e7866bd255f None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:57.426746 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:57.444889 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:57.467890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce684ec-b4fa-453b-82c3-239960e69a37 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:57.504657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5c54b-69b3-4889-a056-c796c966f559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:57.505478 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5c54b-69b3-4889-a056-c796c966f559 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:57.505757 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5c54b-69b3-4889-a056-c796c966f559 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:57.506031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5c54b-69b3-4889-a056-c796c966f559 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:57.512660 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5c54b-69b3-4889-a056-c796c966f559 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:57.514094 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5c54b-69b3-4889-a056-c796c966f559 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:57.515164 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ceb1d-fa25-4767-87c1-8e7866bd255f None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:57.530684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-cbf5c54b-69b3-4889-a056-c796c966f559 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:24:57.530684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:57.530684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:57.530684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:57.530684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:57.530684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:57.530684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:57.530684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:57.530684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:57.530684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:57.530684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:57.530684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:57.530684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:24:57.530684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:24:57.530684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:24:57.531579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:24:57.531579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:57.531579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:57.531579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:57.531579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:57.531579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:57.531579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:57.531579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:57.531579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:57.531579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:57.531579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:57.531579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:57.531579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:57.531579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:57.531579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:57.531579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:57.531579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:57.532510 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:24:57.532510 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:24:57.532510 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:24:57.532510 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:24:57.532510 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:24:57.532510 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:24:57.532510 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:24:57.532510 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:57.532510 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 300/599] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:24:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:57.533015 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5c54b-69b3-4889-a056-c796c966f559 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:57.556271 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fab08b-ac11-4d74-8d1a-f9a5fdf84153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:57.556998 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fab08b-ac11-4d74-8d1a-f9a5fdf84153 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:57.557283 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fab08b-ac11-4d74-8d1a-f9a5fdf84153 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:57.557559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fab08b-ac11-4d74-8d1a-f9a5fdf84153 None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:57.558340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fab08b-ac11-4d74-8d1a-f9a5fdf84153 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:57.558643 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fab08b-ac11-4d74-8d1a-f9a5fdf84153 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:57.560089 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fab08b-ac11-4d74-8d1a-f9a5fdf84153 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:57.575818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 300/600] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:24:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:57.575818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47563b-f575-467a-9f26-477e3c49054d None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:57.575818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47563b-f575-467a-9f26-477e3c49054d None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:57.575818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47563b-f575-467a-9f26-477e3c49054d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:57.590067 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-6f47563b-f575-467a-9f26-477e3c49054d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:57.591251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-6f47563b-f575-467a-9f26-477e3c49054d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:57.643595 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5c54b-69b3-4889-a056-c796c966f559 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:57.663827 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 301/601] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:24:57.663827 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47563b-f575-467a-9f26-477e3c49054d None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:57.671759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:57.684390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020b211-a04b-4ed1-b931-b292e3b72fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:57.685475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020b211-a04b-4ed1-b931-b292e3b72fed None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:57.685926 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020b211-a04b-4ed1-b931-b292e3b72fed None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:57.686370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020b211-a04b-4ed1-b931-b292e3b72fed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:57.696994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7020b211-a04b-4ed1-b931-b292e3b72fed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:57.698422 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7020b211-a04b-4ed1-b931-b292e3b72fed None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:57.702554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:57.717942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-7020b211-a04b-4ed1-b931-b292e3b72fed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:57.717942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:57.717942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:57.717942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:57.717942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:57.717942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:57.717942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:57.717942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:57.717942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:57.717942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:57.717942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:57.717942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:57.717942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:57.717942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:57.717942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:57.718877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:57.718877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:57.718877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:57.718877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:57.718877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:57.718877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:57.718877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:57.718877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:57.718877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:57.718877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:57.718877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:57.718877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:57.718877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:57.718877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:57.718877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:57.718877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:57.718877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:57.724827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:57.724827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:57.724827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:57.724827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:57.724827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 301/602] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:57.724827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020b211-a04b-4ed1-b931-b292e3b72fed None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:57.798355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47563b-f575-467a-9f26-477e3c49054d None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:57.832299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020b211-a04b-4ed1-b931-b292e3b72fed None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:57.836982 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c02481-e92e-4292-b0de-b1333608dd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:57.837874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c02481-e92e-4292-b0de-b1333608dd19 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:57.838160 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c02481-e92e-4292-b0de-b1333608dd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:57.838437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c02481-e92e-4292-b0de-b1333608dd19 None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:57.839223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c02481-e92e-4292-b0de-b1333608dd19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:57.839529 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c02481-e92e-4292-b0de-b1333608dd19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:57.841417 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c02481-e92e-4292-b0de-b1333608dd19 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:57.860364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 302/603] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:24:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:57.860364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c02481-e92e-4292-b0de-b1333608dd19 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:57.874082 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cce361-b045-4c22-a190-5a07bf24bab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:57.874979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cce361-b045-4c22-a190-5a07bf24bab6 None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:57.875268 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cce361-b045-4c22-a190-5a07bf24bab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:57.875546 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cce361-b045-4c22-a190-5a07bf24bab6 None admin] PATH_INFO: `/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/2b8836193f1a47e1a62ea5167e01b26e/roles/95eccd30e39b4105ac6e8c0d9f973951` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:57.891913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cce361-b045-4c22-a190-5a07bf24bab6 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f082d7ed3694fd9998623f3d4a5fea9, user_id=2b8836193f1a47e1a62ea5167e01b26e, role_id=95eccd30e39b4105ac6e8c0d9f973951)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:57.893491 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cce361-b045-4c22-a190-5a07bf24bab6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:57.919816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 302/604] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:24:57 2022] PUT /identity/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/2b8836193f1a47e1a62ea5167e01b26e/roles/95eccd30e39b4105ac6e8c0d9f973951 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:24:57.919816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cce361-b045-4c22-a190-5a07bf24bab6 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:57.955135 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:57.968506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:57.991544 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c02481-e92e-4292-b0de-b1333608dd19 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:58.031895 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ae466-c03c-4ca4-8a1a-c6ecc9c0d927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:58.032809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ae466-c03c-4ca4-8a1a-c6ecc9c0d927 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:58.033212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ae466-c03c-4ca4-8a1a-c6ecc9c0d927 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:58.033593 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ae466-c03c-4ca4-8a1a-c6ecc9c0d927 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:58.046313 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ae466-c03c-4ca4-8a1a-c6ecc9c0d927 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:58.054651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-349ae466-c03c-4ca4-8a1a-c6ecc9c0d927 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:58.054651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:58.054651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:58.054651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:58.054651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:58.054651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:58.054651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:58.054651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:58.054651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:58.054651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:58.054651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:58.054651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:58.054651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:58.054651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:58.054651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:58.054651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:58.055661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:58.055661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:58.055661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:58.055661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:58.055661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:58.055661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:58.055661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:58.055661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:58.055661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:58.055661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:58.055661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:58.055661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:58.055661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:58.055661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:58.055661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:58.055661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:58.056559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:58.056559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:58.056559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:58.056559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:58.056559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:58.056559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:58.056559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:58.056559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:58.056559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:58.056559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:58.056559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:58.056559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:58.057222 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cce361-b045-4c22-a190-5a07bf24bab6 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:58.057297 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 303/605] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:24:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:58.057297 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ae466-c03c-4ca4-8a1a-c6ecc9c0d927 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:58.095164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58139657-039c-4e64-bfc9-a8e787cbfca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:58.095858 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58139657-039c-4e64-bfc9-a8e787cbfca0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:58.096035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58139657-039c-4e64-bfc9-a8e787cbfca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:58.096319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58139657-039c-4e64-bfc9-a8e787cbfca0 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:58.097100 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58139657-039c-4e64-bfc9-a8e787cbfca0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:58.097404 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58139657-039c-4e64-bfc9-a8e787cbfca0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:58.098879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58139657-039c-4e64-bfc9-a8e787cbfca0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:58.116341 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 303/606] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:58.117390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58139657-039c-4e64-bfc9-a8e787cbfca0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:58.180805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ae466-c03c-4ca4-8a1a-c6ecc9c0d927 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:58.218285 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:58.235259 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58139657-039c-4e64-bfc9-a8e787cbfca0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:58.238888 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a99f7-8067-43c1-b10a-d6f05dc2bff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:58.240068 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a99f7-8067-43c1-b10a-d6f05dc2bff4 None admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:58.240558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a99f7-8067-43c1-b10a-d6f05dc2bff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:58.241063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a99f7-8067-43c1-b10a-d6f05dc2bff4 None admin] PATH_INFO: `/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/1a67e007fc314b34aac2471a7c20f240/roles/95eccd30e39b4105ac6e8c0d9f973951` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:58.262573 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a99f7-8067-43c1-b10a-d6f05dc2bff4 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f082d7ed3694fd9998623f3d4a5fea9, user_id=1a67e007fc314b34aac2471a7c20f240, role_id=95eccd30e39b4105ac6e8c0d9f973951)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:58.265032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a99f7-8067-43c1-b10a-d6f05dc2bff4 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:58.270170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:58.273417 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f62b6-842e-4625-9a34-8299dad8e1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:58.275022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f62b6-842e-4625-9a34-8299dad8e1ce None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:58.275348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f62b6-842e-4625-9a34-8299dad8e1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:58.275699 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f62b6-842e-4625-9a34-8299dad8e1ce None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:58.277711 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f62b6-842e-4625-9a34-8299dad8e1ce None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:58.277711 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f62b6-842e-4625-9a34-8299dad8e1ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:58.279049 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f62b6-842e-4625-9a34-8299dad8e1ce None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:58.290471 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 304/608] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:24:58 2022] PUT /identity/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/1a67e007fc314b34aac2471a7c20f240/roles/95eccd30e39b4105ac6e8c0d9f973951 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:24:58.290642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 304/608] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jun 6 10:24:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:58.296103 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f62b6-842e-4625-9a34-8299dad8e1ce None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:58.298414 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a99f7-8067-43c1-b10a-d6f05dc2bff4 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:58.501912 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f62b6-842e-4625-9a34-8299dad8e1ce None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:58.518849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a99f7-8067-43c1-b10a-d6f05dc2bff4 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:58.580855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e893c-25fc-4c64-b4f1-0e37874830ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:58.582094 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e893c-25fc-4c64-b4f1-0e37874830ab None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:58.582382 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e893c-25fc-4c64-b4f1-0e37874830ab None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:58.582653 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e893c-25fc-4c64-b4f1-0e37874830ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:58.584420 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f313a9-3ce9-4bdc-b6df-27cc4b788daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:58.585466 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f313a9-3ce9-4bdc-b6df-27cc4b788daf None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:58.586046 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f313a9-3ce9-4bdc-b6df-27cc4b788daf None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:58.586533 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f313a9-3ce9-4bdc-b6df-27cc4b788daf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:58.595576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e893c-25fc-4c64-b4f1-0e37874830ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:58.596590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f313a9-3ce9-4bdc-b6df-27cc4b788daf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:58.597117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e893c-25fc-4c64-b4f1-0e37874830ab None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:58.599244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f313a9-3ce9-4bdc-b6df-27cc4b788daf None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:58.616729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-92f313a9-3ce9-4bdc-b6df-27cc4b788daf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:24:58.616729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:58.616729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:58.616729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:58.616729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:58.616729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:58.616729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:58.616729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:58.616729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:58.616729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:58.616729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:58.616729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:58.616729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:24:58.616729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:24:58.616729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:24:58.617688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:24:58.617688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:58.617688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:58.617688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:58.617688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:58.617688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:58.617688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:58.617688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:58.617688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:58.617688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:58.617688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:58.617688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:58.617688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:58.617688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:58.617688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:58.617688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:58.617688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:58.618603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:24:58.618603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:24:58.618603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:24:58.618603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:24:58.618603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:24:58.618603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:24:58.618603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:24:58.618603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:58.619029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-0c5e893c-25fc-4c64-b4f1-0e37874830ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:58.619029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:58.619029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:58.619029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:58.619029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:58.619029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:58.619029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:58.619029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:58.619029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:58.619029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:58.619029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:58.619029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:58.619029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:24:58.619029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:24:58.619029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:24:58.619852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:24:58.619852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:58.619852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:58.619852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:58.619852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:58.619852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:58.619852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:58.619852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:58.619852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:58.619852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:58.619852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:58.619852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:58.619852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:58.619852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:58.619852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:58.619852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:58.619852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:58.620769 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:24:58.620769 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:24:58.620769 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:24:58.620769 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:58.620769 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 305/609] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:58.623284 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 305/610] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:24:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:58.626472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e893c-25fc-4c64-b4f1-0e37874830ab None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:58.630990 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f313a9-3ce9-4bdc-b6df-27cc4b788daf None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:58.644634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:58.687801 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:58.762597 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e893c-25fc-4c64-b4f1-0e37874830ab None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:58.764337 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f313a9-3ce9-4bdc-b6df-27cc4b788daf None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:58.807626 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d6232-ed6f-4924-b432-d1b9a43b9270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:58.807626 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d6232-ed6f-4924-b432-d1b9a43b9270 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:58.807626 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d6232-ed6f-4924-b432-d1b9a43b9270 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:58.807626 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d6232-ed6f-4924-b432-d1b9a43b9270 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:58.808065 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d6232-ed6f-4924-b432-d1b9a43b9270 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:58.808172 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d6232-ed6f-4924-b432-d1b9a43b9270 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:58.809759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d6232-ed6f-4924-b432-d1b9a43b9270 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:58.825465 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 306/611] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:24:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:58.832235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d6232-ed6f-4924-b432-d1b9a43b9270 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:58.865636 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d30351-560e-4b49-8307-7525401c2a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:58.866586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d30351-560e-4b49-8307-7525401c2a3e None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:58.867011 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d30351-560e-4b49-8307-7525401c2a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:58.867426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d30351-560e-4b49-8307-7525401c2a3e None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:58.868318 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d30351-560e-4b49-8307-7525401c2a3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:58.868764 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d30351-560e-4b49-8307-7525401c2a3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:58.870284 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d30351-560e-4b49-8307-7525401c2a3e None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:58.902399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 306/612] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:58.907856 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d30351-560e-4b49-8307-7525401c2a3e None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:58.909028 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:58.950326 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d6232-ed6f-4924-b432-d1b9a43b9270 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:58.992624 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265baebb-799b-45d6-9160-8a9d4709db60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:58.994256 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265baebb-799b-45d6-9160-8a9d4709db60 None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:58.994692 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265baebb-799b-45d6-9160-8a9d4709db60 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:58.995115 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265baebb-799b-45d6-9160-8a9d4709db60 None admin] PATH_INFO: `/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/881069f9b40a485dac2dda684dd6cd3b/roles/95eccd30e39b4105ac6e8c0d9f973951` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:59.016496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265baebb-799b-45d6-9160-8a9d4709db60 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f082d7ed3694fd9998623f3d4a5fea9, user_id=881069f9b40a485dac2dda684dd6cd3b, role_id=95eccd30e39b4105ac6e8c0d9f973951)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:59.018474 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265baebb-799b-45d6-9160-8a9d4709db60 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:59.036850 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 307/613] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:24:58 2022] PUT /identity/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/881069f9b40a485dac2dda684dd6cd3b/roles/95eccd30e39b4105ac6e8c0d9f973951 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:24:59.044015 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05477c-e2d9-4f66-97fe-f511e8c02d29 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:59.044015 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05477c-e2d9-4f66-97fe-f511e8c02d29 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:59.044015 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05477c-e2d9-4f66-97fe-f511e8c02d29 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:59.049450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 308/614] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:24:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:24:59.051082 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162a56a-3dc8-4af4-901e-5f0f4a13d9b6 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:59.051370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162a56a-3dc8-4af4-901e-5f0f4a13d9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:59.051652 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162a56a-3dc8-4af4-901e-5f0f4a13d9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:59.088510 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d30351-560e-4b49-8307-7525401c2a3e None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:59.090057 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9162a56a-3dc8-4af4-901e-5f0f4a13d9b6 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:24:59.096698 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9162a56a-3dc8-4af4-901e-5f0f4a13d9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:24:59.110299 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:59.175353 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6c663-2ef0-422d-aacb-6acadc7b3bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:59.176506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6c663-2ef0-422d-aacb-6acadc7b3bca None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:59.176849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6c663-2ef0-422d-aacb-6acadc7b3bca None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:59.177123 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6c663-2ef0-422d-aacb-6acadc7b3bca None admin] PATH_INFO: `/v3/users/nova` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:59.196096 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6c663-2ef0-422d-aacb-6acadc7b3bca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:59.197569 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6c663-2ef0-422d-aacb-6acadc7b3bca None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:59.225148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-9cd6c663-2ef0-422d-aacb-6acadc7b3bca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 10:24:59.225148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:59.225148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:59.225148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:59.225148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:59.225148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:59.225148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:59.225148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:59.225148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:59.225148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:59.225148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:59.225148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:59.225148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:24:59.225148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:24:59.225148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:24:59.225148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:24:59.226153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:59.226153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:59.226153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:24:59.226153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:24:59.226153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:24:59.226153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:24:59.226153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:59.226153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:59.226153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:59.226153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:59.226153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:59.226153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:59.226153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:59.226153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:59.226153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:59.226153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:59.226153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:59.226153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:59.227080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:59.227080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:59.227080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:24:59.227080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:24:59.227080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:24:59.227080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:24:59.227080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:24:59.227080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:24:59.227080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 10:24:59.227080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:24:59.227080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 307/615] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jun 6 10:24:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:24:59.242008 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6c663-2ef0-422d-aacb-6acadc7b3bca None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:59.243882 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 309/616] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:24:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 10:24:59.277393 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162a56a-3dc8-4af4-901e-5f0f4a13d9b6 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:59.341664 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:59.386368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6c663-2ef0-422d-aacb-6acadc7b3bca None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:59.411023 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162a56a-3dc8-4af4-901e-5f0f4a13d9b6 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:59.434314 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fec495-5b71-43dd-87b0-441618f472a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:59.435263 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fec495-5b71-43dd-87b0-441618f472a5 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:59.435548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fec495-5b71-43dd-87b0-441618f472a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:59.436112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fec495-5b71-43dd-87b0-441618f472a5 None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:59.440274 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fec495-5b71-43dd-87b0-441618f472a5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:59.440590 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fec495-5b71-43dd-87b0-441618f472a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:59.443033 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fec495-5b71-43dd-87b0-441618f472a5 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:59.468139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 308/617] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jun 6 10:24:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:24:59.474652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fec495-5b71-43dd-87b0-441618f472a5 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:59.501071 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ffd7b-75b8-4135-87e8-ed8777ec61be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:59.501885 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ffd7b-75b8-4135-87e8-ed8777ec61be None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:59.502166 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ffd7b-75b8-4135-87e8-ed8777ec61be None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:59.502435 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ffd7b-75b8-4135-87e8-ed8777ec61be None admin] PATH_INFO: `/v3/roles/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:59.504027 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:59.540543 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ffd7b-75b8-4135-87e8-ed8777ec61be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:24:59.548443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-5a5ffd7b-75b8-4135-87e8-ed8777ec61be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:59.548443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:24:59.548443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:24:59.548443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:24:59.548443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:24:59.548443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:24:59.548443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:24:59.548443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:24:59.548443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:24:59.548443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:24:59.548443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:24:59.548443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:24:59.548443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:24:59.548443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:24:59.548443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:24:59.548443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:24:59.551069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:24:59.551069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:24:59.551069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:24:59.551069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:24:59.551069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:24:59.551069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:24:59.551069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:24:59.551069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:24:59.551069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:24:59.551069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:24:59.551069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:24:59.551069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:24:59.551069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:24:59.551069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:24:59.551069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:24:59.551069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:24:59.553560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:24:59.553560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:24:59.553560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:24:59.553560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:24:59.553560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:24:59.553560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:24:59.553560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:24:59.553560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:24:59.553560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:24:59.553560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:24:59.553560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:24:59.553560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:24:59.553560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 310/618] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:24:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:24:59.557404 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ffd7b-75b8-4135-87e8-ed8777ec61be None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:59.589996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fec495-5b71-43dd-87b0-441618f472a5 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:59.627288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a845af-1dc3-4e16-b9da-95e76e0bf8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:59.628845 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a845af-1dc3-4e16-b9da-95e76e0bf8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:59.629395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a845af-1dc3-4e16-b9da-95e76e0bf8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:59.629962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a845af-1dc3-4e16-b9da-95e76e0bf8e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:59.704117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a845af-1dc3-4e16-b9da-95e76e0bf8e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:59.705911 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a845af-1dc3-4e16-b9da-95e76e0bf8e4 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:59.736885 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:59.804125 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ffd7b-75b8-4135-87e8-ed8777ec61be None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:59.811478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 309/619] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:24:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 06 10:24:59.818367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:59.822248 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a845af-1dc3-4e16-b9da-95e76e0bf8e4 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:59.896511 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15646a-e1eb-48a3-a816-9b88dc55f0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:59.897458 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15646a-e1eb-48a3-a816-9b88dc55f0c3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:59.897918 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15646a-e1eb-48a3-a816-9b88dc55f0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:59.898345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15646a-e1eb-48a3-a816-9b88dc55f0c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:59.899494 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15646a-e1eb-48a3-a816-9b88dc55f0c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:24:59.899974 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15646a-e1eb-48a3-a816-9b88dc55f0c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:59.901523 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15646a-e1eb-48a3-a816-9b88dc55f0c3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:24:59.926829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 311/620] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jun 6 10:24:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:24:59.929885 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a845af-1dc3-4e16-b9da-95e76e0bf8e4 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:24:59.932613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15646a-e1eb-48a3-a816-9b88dc55f0c3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:24:59.964266 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:24:59.968425 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da93242-2a6c-46f9-b44d-83a0c08251f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:24:59.969316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da93242-2a6c-46f9-b44d-83a0c08251f3 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:24:59.969664 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da93242-2a6c-46f9-b44d-83a0c08251f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:24:59.970029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da93242-2a6c-46f9-b44d-83a0c08251f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:24:59.982317 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da93242-2a6c-46f9-b44d-83a0c08251f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:24:59.984249 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da93242-2a6c-46f9-b44d-83a0c08251f3 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:00.009237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-1da93242-2a6c-46f9-b44d-83a0c08251f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:00.009237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:00.009237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:00.009237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:00.009237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:00.009237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:00.009237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:00.009237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:00.009237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:00.009237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:00.009237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:00.009237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:00.009237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:00.009237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:00.009237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:00.010221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:00.010221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:00.010221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:00.010221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:00.010221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:00.010221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:00.010221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:00.010221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:00.010221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:00.010221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:00.010221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:00.010221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:00.010221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:00.010221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:00.010221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:00.010221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:00.010221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:00.011121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:00.011121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:00.011121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:00.011121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:00.013509 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 310/621] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:00.017581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da93242-2a6c-46f9-b44d-83a0c08251f3 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:00.142379 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da93242-2a6c-46f9-b44d-83a0c08251f3 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:00.186451 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1991360-9a57-4d49-816c-9ec8bad9ad96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:00.187126 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1991360-9a57-4d49-816c-9ec8bad9ad96 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:00.187409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1991360-9a57-4d49-816c-9ec8bad9ad96 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:00.187823 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1991360-9a57-4d49-816c-9ec8bad9ad96 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:00.188789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1991360-9a57-4d49-816c-9ec8bad9ad96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:00.189231 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1991360-9a57-4d49-816c-9ec8bad9ad96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:00.191857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1991360-9a57-4d49-816c-9ec8bad9ad96 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:00.205077 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 311/622] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:00.217038 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1991360-9a57-4d49-816c-9ec8bad9ad96 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:00.229793 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15646a-e1eb-48a3-a816-9b88dc55f0c3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:00.271228 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1960e-4eaf-4aaa-87a1-61d6ced9402e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:00.273374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1960e-4eaf-4aaa-87a1-61d6ced9402e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:00.273374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1960e-4eaf-4aaa-87a1-61d6ced9402e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:00.273374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1960e-4eaf-4aaa-87a1-61d6ced9402e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:00.283809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:00.299811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1960e-4eaf-4aaa-87a1-61d6ced9402e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:00.299811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1960e-4eaf-4aaa-87a1-61d6ced9402e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:00.331813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-a2b1960e-4eaf-4aaa-87a1-61d6ced9402e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:00.331813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:00.331813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:00.331813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:00.331813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:00.331813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:00.331813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:00.331813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:00.331813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:00.331813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:00.331813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:00.331813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:00.331813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:00.331813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:00.331813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:00.333924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:00.333924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:00.333924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:00.333924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:00.333924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:00.333924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:00.333924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:00.333924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:00.333924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:00.333924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:00.333924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:00.333924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:00.333924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:00.333924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:00.333924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:00.333924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:00.333924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:00.336335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:00.336335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:00.336335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:00.336335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:00.339814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 312/623] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:24:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:00.339814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1960e-4eaf-4aaa-87a1-61d6ced9402e None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:00.394800 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:00.409994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1991360-9a57-4d49-816c-9ec8bad9ad96 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:00.495210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a1b7b-6cde-4c43-a25a-fa70cf895d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:00.496072 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a1b7b-6cde-4c43-a25a-fa70cf895d0b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:00.496351 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a1b7b-6cde-4c43-a25a-fa70cf895d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:00.496621 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a1b7b-6cde-4c43-a25a-fa70cf895d0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:00.511882 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1960e-4eaf-4aaa-87a1-61d6ced9402e None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:00.513026 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a1b7b-6cde-4c43-a25a-fa70cf895d0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:00.514705 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a1b7b-6cde-4c43-a25a-fa70cf895d0b None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:00.548205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-8a7a1b7b-6cde-4c43-a25a-fa70cf895d0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:00.548205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:00.548205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:00.548205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:00.548205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:00.548205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:00.548205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:00.548205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:00.548205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:00.548205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:00.548205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:00.548205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:00.548205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:00.548205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:00.548205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:00.550831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:00.550831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:00.550831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:00.550831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:00.550831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:00.550831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:00.550831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:00.550831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:00.550831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:00.550831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:00.550831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:00.550831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:00.550831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:00.550831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:00.550831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:00.550831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:00.550831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:00.553003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:00.553003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:00.553003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:00.553003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:00.553003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:00.553003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:00.553003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:00.553003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:00.553003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 312/624] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:00.559823 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a1b7b-6cde-4c43-a25a-fa70cf895d0b None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:00.561238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ed84d-988c-45d7-92d8-aca7ef425b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:00.561919 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ed84d-988c-45d7-92d8-aca7ef425b31 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:00.562202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ed84d-988c-45d7-92d8-aca7ef425b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:00.562482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ed84d-988c-45d7-92d8-aca7ef425b31 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:00.563207 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ed84d-988c-45d7-92d8-aca7ef425b31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:00.563513 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ed84d-988c-45d7-92d8-aca7ef425b31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:00.564913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ed84d-988c-45d7-92d8-aca7ef425b31 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:00.590204 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 313/625] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:00.591550 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f24e5-0baa-4f7e-bc67-105921ed7b0b None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:00.591864 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f24e5-0baa-4f7e-bc67-105921ed7b0b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:00.592178 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f24e5-0baa-4f7e-bc67-105921ed7b0b None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:00.593665 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 314/626] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:00.595279 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f24e5-0baa-4f7e-bc67-105921ed7b0b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:00.687865 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:00.698604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:00.704106 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a1b7b-6cde-4c43-a25a-fa70cf895d0b None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:00.713142 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f24e5-0baa-4f7e-bc67-105921ed7b0b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:00.749896 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a0490-3749-426b-ac79-53113180c856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:00.750943 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a0490-3749-426b-ac79-53113180c856 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:00.751221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a0490-3749-426b-ac79-53113180c856 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:00.751490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a0490-3749-426b-ac79-53113180c856 None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:00.752664 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a0490-3749-426b-ac79-53113180c856 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:00.752973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a0490-3749-426b-ac79-53113180c856 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:00.754738 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a0490-3749-426b-ac79-53113180c856 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:00.757854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e09a4-0f82-4d15-9855-fd48c56b85c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:00.758913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e09a4-0f82-4d15-9855-fd48c56b85c6 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:00.759394 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e09a4-0f82-4d15-9855-fd48c56b85c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:00.759906 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e09a4-0f82-4d15-9855-fd48c56b85c6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:00.768328 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 313/627] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:25:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:00.770595 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e09a4-0f82-4d15-9855-fd48c56b85c6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:00.770802 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82522e90-c940-44f0-a7b1-10c60d60e5b0 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:00.771048 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82522e90-c940-44f0-a7b1-10c60d60e5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:00.771323 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82522e90-c940-44f0-a7b1-10c60d60e5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:00.773145 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e09a4-0f82-4d15-9855-fd48c56b85c6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:00.790788 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-82522e90-c940-44f0-a7b1-10c60d60e5b0 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:00.792165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-82522e90-c940-44f0-a7b1-10c60d60e5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:00.793613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-6f6e09a4-0f82-4d15-9855-fd48c56b85c6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 10:25:00.793613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:00.793613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:00.793613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:00.793613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:00.793613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:00.793613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:00.793613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:00.793613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:00.793613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:00.793613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:00.793613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:00.793613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:25:00.793613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:25:00.793613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:25:00.793613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:25:00.794632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:00.794632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:00.794632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:00.794632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:00.794632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:00.794632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:00.794632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:00.794632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:00.794632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:00.794632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:00.794632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:00.794632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:00.794632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:00.794632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:00.794632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:00.794632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:00.794632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:00.794632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:00.795568 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:00.795568 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:00.795568 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:25:00.795568 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:25:00.795568 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:25:00.795568 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:25:00.795568 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:25:00.795568 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:25:00.795568 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 10:25:00.795568 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:00.803595 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 315/628] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:25:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:00.803595 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e09a4-0f82-4d15-9855-fd48c56b85c6 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:00.883746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 314/629] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:25:00.885094 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82522e90-c940-44f0-a7b1-10c60d60e5b0 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:00.910069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e09a4-0f82-4d15-9855-fd48c56b85c6 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:00.947355 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e6d9c-2213-4ce6-a18c-e4c017668cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:00.948108 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e6d9c-2213-4ce6-a18c-e4c017668cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:00.948390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e6d9c-2213-4ce6-a18c-e4c017668cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:00.948663 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e6d9c-2213-4ce6-a18c-e4c017668cd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:00.949542 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e6d9c-2213-4ce6-a18c-e4c017668cd2 None admin] RBAC: Adding query filter params (role.id=95eccd30e39b4105ac6e8c0d9f973951, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=ed8a40e4f1034a8e99d0eb505e448db9) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:00.949848 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e6d9c-2213-4ce6-a18c-e4c017668cd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:00.951317 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e6d9c-2213-4ce6-a18c-e4c017668cd2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:00.958313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 316/630] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:25:00 2022] GET /identity/v3/role_assignments?user.id=ed8a40e4f1034a8e99d0eb505e448db9&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=95eccd30e39b4105ac6e8c0d9f973951 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:00.959216 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e6d9c-2213-4ce6-a18c-e4c017668cd2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:00.968627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:01.011390 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:01.027370 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82522e90-c940-44f0-a7b1-10c60d60e5b0 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:01.066900 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9ec0b-cbfc-4114-8a68-c33fd10960c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:01.068083 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9ec0b-cbfc-4114-8a68-c33fd10960c1 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:01.068592 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9ec0b-cbfc-4114-8a68-c33fd10960c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:01.069126 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9ec0b-cbfc-4114-8a68-c33fd10960c1 None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:01.070228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9ec0b-cbfc-4114-8a68-c33fd10960c1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:01.070770 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9ec0b-cbfc-4114-8a68-c33fd10960c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:01.072731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9ec0b-cbfc-4114-8a68-c33fd10960c1 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:01.083434 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e6d9c-2213-4ce6-a18c-e4c017668cd2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:01.095093 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 315/631] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Jun 6 10:25:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:01.101114 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9ec0b-cbfc-4114-8a68-c33fd10960c1 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:01.136660 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f77008-a2e3-4a7f-a564-e669fcd9a9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:01.137605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f77008-a2e3-4a7f-a564-e669fcd9a9da None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:01.137816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f77008-a2e3-4a7f-a564-e669fcd9a9da None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:01.138094 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f77008-a2e3-4a7f-a564-e669fcd9a9da None admin] PATH_INFO: `/v3/roles/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:01.155295 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f77008-a2e3-4a7f-a564-e669fcd9a9da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:25:01.163714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-f4f77008-a2e3-4a7f-a564-e669fcd9a9da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:25:01.163714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:01.163714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:01.163714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:01.163714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:01.163714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:01.163714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:01.163714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:01.163714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:01.163714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:01.163714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:01.163714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:01.163714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:25:01.163714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:25:01.163714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:25:01.163714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:25:01.164820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:25:01.164820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:25:01.164820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:25:01.164820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:25:01.164820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:01.164820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:01.164820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:01.164820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:01.164820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:01.164820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:01.164820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:01.164820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:01.164820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:01.164820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:01.164820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:01.164820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:01.165643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:01.165643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:01.165643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:01.165643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:01.165643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:25:01.165643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:25:01.165643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:25:01.165643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:25:01.165643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:25:01.165643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:25:01.165643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:25:01.165643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:01.170456 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 317/632] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:25:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:01.171479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce89957-0b9e-4373-9da9-754246c26bf9 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:01.171777 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce89957-0b9e-4373-9da9-754246c26bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:01.172080 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce89957-0b9e-4373-9da9-754246c26bf9 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:01.173655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 318/633] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:01.174266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce89957-0b9e-4373-9da9-754246c26bf9 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:01.231445 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9ec0b-cbfc-4114-8a68-c33fd10960c1 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:01.275531 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd5f01-fbe1-440b-8c06-c67b0c61dcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:01.276725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd5f01-fbe1-440b-8c06-c67b0c61dcac None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:01.277013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd5f01-fbe1-440b-8c06-c67b0c61dcac None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:01.277288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd5f01-fbe1-440b-8c06-c67b0c61dcac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:01.287367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:01.290332 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd5f01-fbe1-440b-8c06-c67b0c61dcac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:01.292061 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd5f01-fbe1-440b-8c06-c67b0c61dcac None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:01.302056 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:01.311668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-eacd5f01-fbe1-440b-8c06-c67b0c61dcac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:01.311668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:01.311668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:01.311668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:01.311668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:01.311668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:01.311668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:01.311668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:01.311668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:01.311668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:01.311668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:01.311668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:01.311668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:01.311668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:01.311668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:01.312604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:01.312604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:01.312604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:01.312604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:01.312604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:01.312604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:01.312604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:01.312604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:01.312604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:01.312604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:01.312604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:01.312604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:01.312604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:01.312604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:01.312604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:01.312604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:01.312604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:01.313484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:01.313484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:01.313484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:01.313484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:01.313484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 316/634] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:01.314547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580334df-40cc-4d3a-a0f5-ba6d371e51a3 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:01.315168 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580334df-40cc-4d3a-a0f5-ba6d371e51a3 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:01.315168 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580334df-40cc-4d3a-a0f5-ba6d371e51a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:01.329501 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce89957-0b9e-4373-9da9-754246c26bf9 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:01.339982 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-580334df-40cc-4d3a-a0f5-ba6d371e51a3 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:01.341566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-580334df-40cc-4d3a-a0f5-ba6d371e51a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:01.376421 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5f234-a997-4c67-8437-8b40b630b38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:01.380366 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5f234-a997-4c67-8437-8b40b630b38e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:01.381890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5f234-a997-4c67-8437-8b40b630b38e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:01.382444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5f234-a997-4c67-8437-8b40b630b38e None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:01.384524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5f234-a997-4c67-8437-8b40b630b38e None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:01.385284 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5f234-a997-4c67-8437-8b40b630b38e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:01.389563 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5f234-a997-4c67-8437-8b40b630b38e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:01.402029 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 319/635] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jun 6 10:25:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:01.403378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5f234-a997-4c67-8437-8b40b630b38e None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:01.437161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 317/636] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:25:01.438087 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580334df-40cc-4d3a-a0f5-ba6d371e51a3 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:01.525779 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5f234-a997-4c67-8437-8b40b630b38e None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:01.549056 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580334df-40cc-4d3a-a0f5-ba6d371e51a3 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:01.558753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:01.569328 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99cc33-6ae7-4a39-8f5a-fe0e2ff69895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:01.569997 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99cc33-6ae7-4a39-8f5a-fe0e2ff69895 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:01.570417 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99cc33-6ae7-4a39-8f5a-fe0e2ff69895 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:01.570822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99cc33-6ae7-4a39-8f5a-fe0e2ff69895 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:01.571548 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99cc33-6ae7-4a39-8f5a-fe0e2ff69895 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:01.571882 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99cc33-6ae7-4a39-8f5a-fe0e2ff69895 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:01.573424 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99cc33-6ae7-4a39-8f5a-fe0e2ff69895 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:01.583963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:01.593277 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 320/637] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:01.593277 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99cc33-6ae7-4a39-8f5a-fe0e2ff69895 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:01.599187 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46009c8c-41be-48fd-a28a-afd1f5595e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:01.600436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46009c8c-41be-48fd-a28a-afd1f5595e90 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:01.600930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46009c8c-41be-48fd-a28a-afd1f5595e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:01.601434 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46009c8c-41be-48fd-a28a-afd1f5595e90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:01.614083 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46009c8c-41be-48fd-a28a-afd1f5595e90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:01.615969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46009c8c-41be-48fd-a28a-afd1f5595e90 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:01.637478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-46009c8c-41be-48fd-a28a-afd1f5595e90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:01.637478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:01.637478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:01.637478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:01.637478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:01.637478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:01.637478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:01.637478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:01.637478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:01.637478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:01.637478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:01.637478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:01.637478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:01.637478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:01.637478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:01.638427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:01.638427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:01.638427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:01.638427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:01.638427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:01.638427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:01.638427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:01.638427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:01.638427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:01.638427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:01.638427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:01.638427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:01.638427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:01.638427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:01.638427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:01.638427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:01.638427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:01.639311 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:01.639311 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:01.639311 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:01.639311 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:01.641757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 318/638] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:01.642811 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46009c8c-41be-48fd-a28a-afd1f5595e90 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:01.713730 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99cc33-6ae7-4a39-8f5a-fe0e2ff69895 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:01.751494 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b70000-ecb4-46d4-89e9-83c9ed1a8fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:01.752999 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b70000-ecb4-46d4-89e9-83c9ed1a8fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:01.753815 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b70000-ecb4-46d4-89e9-83c9ed1a8fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:01.754093 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b70000-ecb4-46d4-89e9-83c9ed1a8fa3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:01.759203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46009c8c-41be-48fd-a28a-afd1f5595e90 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:01.764838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b70000-ecb4-46d4-89e9-83c9ed1a8fa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:25:01.773873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-38b70000-ecb4-46d4-89e9-83c9ed1a8fa3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:25:01.773873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:01.773873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:01.773873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:01.773873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:01.773873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:01.773873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:01.773873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:01.773873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:01.773873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:01.773873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:01.773873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:01.773873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:25:01.773873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:25:01.773873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:25:01.773873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:25:01.774829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:25:01.774829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:25:01.774829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:25:01.774829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:25:01.774829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:01.774829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:01.774829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:01.774829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:01.774829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:01.774829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:01.774829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:01.774829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:01.774829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:01.774829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:01.774829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:01.774829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:01.775659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:01.775659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:01.775659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:01.775659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:01.775659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:25:01.775659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:25:01.775659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:25:01.775659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:25:01.775659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:25:01.775659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:25:01.775659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:25:01.775659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:01.775659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 321/639] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:25:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:01.776432 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b70000-ecb4-46d4-89e9-83c9ed1a8fa3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:01.802446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a9d48-2aa9-417e-bfe9-02e0705b9dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:01.803282 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a9d48-2aa9-417e-bfe9-02e0705b9dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:01.803559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a9d48-2aa9-417e-bfe9-02e0705b9dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:01.803862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a9d48-2aa9-417e-bfe9-02e0705b9dd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:01.810447 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a9d48-2aa9-417e-bfe9-02e0705b9dd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:01.811975 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a9d48-2aa9-417e-bfe9-02e0705b9dd4 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:01.826513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-7e0a9d48-2aa9-417e-bfe9-02e0705b9dd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:01.826513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:01.826513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:01.826513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:01.826513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:01.826513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:01.826513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:01.826513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:01.826513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:01.826513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:01.826513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:01.826513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:01.826513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:01.826513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:01.826513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:01.827502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:01.827502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:01.827502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:01.827502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:01.827502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:01.827502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:01.827502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:01.827502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:01.827502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:01.827502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:01.827502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:01.827502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:01.827502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:01.827502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:01.827502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:01.827502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:01.827502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:01.828459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:01.828459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:01.828459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:01.828459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:01.828459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:01.828459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:01.828459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:01.828459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:01.828459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 319/640] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:01.829404 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c7e11-f44d-49e5-914d-46a527d38c34 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:01.829404 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c7e11-f44d-49e5-914d-46a527d38c34 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:01.829570 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c7e11-f44d-49e5-914d-46a527d38c34 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:01.830929 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 320/641] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:01.835848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c7e11-f44d-49e5-914d-46a527d38c34 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:01.837664 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:01.901209 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:01.909489 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b70000-ecb4-46d4-89e9-83c9ed1a8fa3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:01.946606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5ef2a-92e5-4f09-93fd-9b30d6578e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:01.947367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5ef2a-92e5-4f09-93fd-9b30d6578e75 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:01.947728 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5ef2a-92e5-4f09-93fd-9b30d6578e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:01.948112 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5ef2a-92e5-4f09-93fd-9b30d6578e75 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:01.948907 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5ef2a-92e5-4f09-93fd-9b30d6578e75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:01.949222 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5ef2a-92e5-4f09-93fd-9b30d6578e75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:01.950593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5ef2a-92e5-4f09-93fd-9b30d6578e75 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:01.959289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c7e11-f44d-49e5-914d-46a527d38c34 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:01.967640 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 322/642] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:01.967640 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5ef2a-92e5-4f09-93fd-9b30d6578e75 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:01.994934 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a245e-36eb-4f28-a714-5630bb6c7133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:01.995590 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a245e-36eb-4f28-a714-5630bb6c7133 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:01.995894 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a245e-36eb-4f28-a714-5630bb6c7133 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:01.996168 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a245e-36eb-4f28-a714-5630bb6c7133 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:01.996903 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a245e-36eb-4f28-a714-5630bb6c7133 None admin] RBAC: Adding query filter params (name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:01.997201 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a245e-36eb-4f28-a714-5630bb6c7133 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:01.998539 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a245e-36eb-4f28-a714-5630bb6c7133 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:02.007658 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 321/643] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jun 6 10:25:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:02.008857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cadfad-25c0-4765-a6aa-70ce9940315a None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:02.009140 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cadfad-25c0-4765-a6aa-70ce9940315a None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:02.009410 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cadfad-25c0-4765-a6aa-70ce9940315a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:02.026026 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-21cadfad-25c0-4765-a6aa-70ce9940315a None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:02.027245 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-21cadfad-25c0-4765-a6aa-70ce9940315a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:02.076920 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5ef2a-92e5-4f09-93fd-9b30d6578e75 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:02.087148 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:02.095833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 322/644] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:25:02.104518 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cadfad-25c0-4765-a6aa-70ce9940315a None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:02.113893 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53369e16-4da4-412d-b087-4b4033ddb4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:02.114559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53369e16-4da4-412d-b087-4b4033ddb4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:02.114845 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53369e16-4da4-412d-b087-4b4033ddb4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:02.115123 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53369e16-4da4-412d-b087-4b4033ddb4c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:02.115949 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53369e16-4da4-412d-b087-4b4033ddb4c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:02.116282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53369e16-4da4-412d-b087-4b4033ddb4c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:02.117689 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53369e16-4da4-412d-b087-4b4033ddb4c7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:02.131819 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 323/645] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:25:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:02.132603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53369e16-4da4-412d-b087-4b4033ddb4c7 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:02.152616 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:02.233847 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cadfad-25c0-4765-a6aa-70ce9940315a None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:02.242118 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53369e16-4da4-412d-b087-4b4033ddb4c7 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:02.273225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6200870-b156-4af9-8f73-5dfc5e175c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:02.274303 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6200870-b156-4af9-8f73-5dfc5e175c12 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:02.274580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6200870-b156-4af9-8f73-5dfc5e175c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:02.274852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6200870-b156-4af9-8f73-5dfc5e175c12 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:02.280026 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11831b15-712b-44e4-a3cd-b5c25a57573d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:02.281082 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11831b15-712b-44e4-a3cd-b5c25a57573d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:02.281545 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11831b15-712b-44e4-a3cd-b5c25a57573d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:02.282019 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11831b15-712b-44e4-a3cd-b5c25a57573d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:02.282885 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6200870-b156-4af9-8f73-5dfc5e175c12 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:02.284908 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6200870-b156-4af9-8f73-5dfc5e175c12 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:02.294512 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11831b15-712b-44e4-a3cd-b5c25a57573d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:02.296509 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11831b15-712b-44e4-a3cd-b5c25a57573d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:02.299828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-e6200870-b156-4af9-8f73-5dfc5e175c12 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 10:25:02.299828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:02.299828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:02.299828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:02.299828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:02.299828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:02.299828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:02.299828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:02.299828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:02.299828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:02.299828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:02.299828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:02.299828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:25:02.299828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:25:02.299828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:25:02.299828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:25:02.300748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:02.300748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:02.300748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:02.300748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:02.300748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:02.300748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:02.300748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:02.300748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:02.300748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:02.300748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:02.300748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:02.300748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:02.300748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:02.300748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:02.300748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:02.300748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:02.300748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:02.300748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:02.301848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:02.301848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:02.301848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:25:02.301848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:25:02.301848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:25:02.301848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:25:02.301848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:25:02.301848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:25:02.301848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 10:25:02.301848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:02.301848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 323/646] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:25:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:02.302907 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6200870-b156-4af9-8f73-5dfc5e175c12 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:02.318485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-11831b15-712b-44e4-a3cd-b5c25a57573d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:02.318485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:02.318485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:02.318485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:02.318485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:02.318485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:02.318485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:02.318485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:02.318485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:02.318485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:02.318485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:02.318485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:02.318485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:02.318485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:02.318485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:02.319531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:02.319531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:02.319531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:02.319531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:02.319531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:02.319531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:02.319531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:02.319531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:02.319531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:02.319531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:02.319531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:02.319531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:02.319531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:02.319531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:02.319531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:02.319531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:02.319531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:02.320482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:02.320482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:02.320482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:02.320482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:02.322342 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 324/647] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:02.327017 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11831b15-712b-44e4-a3cd-b5c25a57573d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:02.345904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:02.411459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:02.421644 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6200870-b156-4af9-8f73-5dfc5e175c12 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:02.434155 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11831b15-712b-44e4-a3cd-b5c25a57573d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:02.459708 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8d5e5-1e44-428e-ab40-36120b792782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:02.460529 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8d5e5-1e44-428e-ab40-36120b792782 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:02.460814 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8d5e5-1e44-428e-ab40-36120b792782 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:02.461090 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8d5e5-1e44-428e-ab40-36120b792782 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:02.468917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de757d74-a852-47ef-9dfe-32bd1673c66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:02.469788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de757d74-a852-47ef-9dfe-32bd1673c66f None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:02.470069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de757d74-a852-47ef-9dfe-32bd1673c66f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:02.470338 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de757d74-a852-47ef-9dfe-32bd1673c66f None admin] PATH_INFO: `/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/fede9e52ad274285b09d748a684ede38/roles/95eccd30e39b4105ac6e8c0d9f973951` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:02.471304 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c8d5e5-1e44-428e-ab40-36120b792782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:25:02.479654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-a8c8d5e5-1e44-428e-ab40-36120b792782 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:25:02.479654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:02.479654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:02.479654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:02.479654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:02.479654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:02.479654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:02.479654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:02.479654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:02.479654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:02.479654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:02.479654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:02.479654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:25:02.479654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:25:02.479654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:25:02.479654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:25:02.480662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:25:02.480662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:25:02.480662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:25:02.480662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:25:02.480662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:02.480662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:02.480662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:02.480662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:02.480662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:02.480662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:02.480662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:02.480662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:02.480662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:02.480662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:02.480662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:02.480662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:02.481498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:02.481498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:02.481498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:02.481498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:02.481498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:25:02.481498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:25:02.481498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:25:02.481498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:25:02.481498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:25:02.481498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:25:02.481498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:25:02.481498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:02.481498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 324/648] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:25:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:02.482808 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8d5e5-1e44-428e-ab40-36120b792782 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:02.490334 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de757d74-a852-47ef-9dfe-32bd1673c66f None admin] RBAC: Authorizing `identity:create_grant(project_id=1f082d7ed3694fd9998623f3d4a5fea9, user_id=fede9e52ad274285b09d748a684ede38, role_id=95eccd30e39b4105ac6e8c0d9f973951)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:02.492468 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de757d74-a852-47ef-9dfe-32bd1673c66f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:02.507352 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 325/649] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:25:02 2022] PUT /identity/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/fede9e52ad274285b09d748a684ede38/roles/95eccd30e39b4105ac6e8c0d9f973951 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:25:02.520872 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de757d74-a852-47ef-9dfe-32bd1673c66f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:02.598630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8d5e5-1e44-428e-ab40-36120b792782 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:02.628780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:02.643492 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77695742-a94a-4779-847a-6a077ce21d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:02.645228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77695742-a94a-4779-847a-6a077ce21d5a None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:02.646034 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77695742-a94a-4779-847a-6a077ce21d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:02.646861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77695742-a94a-4779-847a-6a077ce21d5a None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:02.648713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77695742-a94a-4779-847a-6a077ce21d5a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:02.649147 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77695742-a94a-4779-847a-6a077ce21d5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:02.651016 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77695742-a94a-4779-847a-6a077ce21d5a None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:02.674665 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de757d74-a852-47ef-9dfe-32bd1673c66f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:02.676136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 325/650] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Jun 6 10:25:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:02.677239 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77695742-a94a-4779-847a-6a077ce21d5a None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:02.691599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:02.725391 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea5006-acf4-4561-b084-313648c69896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:02.726237 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea5006-acf4-4561-b084-313648c69896 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:02.726614 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea5006-acf4-4561-b084-313648c69896 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:02.726983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea5006-acf4-4561-b084-313648c69896 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:02.727842 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea5006-acf4-4561-b084-313648c69896 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:02.728246 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea5006-acf4-4561-b084-313648c69896 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:02.729704 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea5006-acf4-4561-b084-313648c69896 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:02.746293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 326/651] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:02.747025 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea5006-acf4-4561-b084-313648c69896 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:02.824357 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77695742-a94a-4779-847a-6a077ce21d5a None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:02.860287 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e24c06-80de-48df-8882-97dee2102b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:02.860873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e24c06-80de-48df-8882-97dee2102b47 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:02.861153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e24c06-80de-48df-8882-97dee2102b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:02.861422 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e24c06-80de-48df-8882-97dee2102b47 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:02.862155 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e24c06-80de-48df-8882-97dee2102b47 None admin] RBAC: Adding query filter params (name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:02.862982 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e24c06-80de-48df-8882-97dee2102b47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:02.864674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e24c06-80de-48df-8882-97dee2102b47 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:02.874155 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 326/652] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jun 6 10:25:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:02.874576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea5006-acf4-4561-b084-313648c69896 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:02.880234 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e24c06-80de-48df-8882-97dee2102b47 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:02.914225 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:02.922200 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a1863-2861-4cc4-8030-128644a36314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:02.923668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a1863-2861-4cc4-8030-128644a36314 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:02.924187 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a1863-2861-4cc4-8030-128644a36314 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:02.924659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a1863-2861-4cc4-8030-128644a36314 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:02.966969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:03.006160 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a1863-2861-4cc4-8030-128644a36314 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:03.006357 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e24c06-80de-48df-8882-97dee2102b47 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:03.008393 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a1863-2861-4cc4-8030-128644a36314 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:03.040870 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc91dcb-5689-4a70-b86e-e472b3b1eea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:03.041666 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc91dcb-5689-4a70-b86e-e472b3b1eea2 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:03.041941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc91dcb-5689-4a70-b86e-e472b3b1eea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:03.042275 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc91dcb-5689-4a70-b86e-e472b3b1eea2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:03.050866 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc91dcb-5689-4a70-b86e-e472b3b1eea2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:03.052461 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc91dcb-5689-4a70-b86e-e472b3b1eea2 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:03.069886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-5bc91dcb-5689-4a70-b86e-e472b3b1eea2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 06 10:25:03.069886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:03.069886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:03.069886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:03.069886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:03.069886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:03.069886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:03.069886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:03.069886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:03.069886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:03.069886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:03.069886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:03.069886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:25:03.069886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:25:03.069886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:25:03.069886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:25:03.070879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:03.070879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:03.070879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:03.070879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:03.070879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:03.070879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:03.070879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:03.070879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:03.070879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:03.070879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:03.070879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:03.070879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:03.070879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:03.070879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:03.070879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:03.070879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:03.070879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:03.070879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:03.071918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:03.071918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:03.071918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:25:03.071918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:25:03.071918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:25:03.071918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:25:03.071918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:25:03.071918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:25:03.071918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 06 10:25:03.071918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:03.075436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 327/653] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:25:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:03.077537 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc91dcb-5689-4a70-b86e-e472b3b1eea2 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:03.151003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 327/654] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:25:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:25:03.151003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a1863-2861-4cc4-8030-128644a36314 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:03.192365 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:03.215777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc91dcb-5689-4a70-b86e-e472b3b1eea2 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:03.258110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:03.264618 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74dcbf0-f1a5-4334-be6f-992a43547ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:03.265755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74dcbf0-f1a5-4334-be6f-992a43547ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:03.266235 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74dcbf0-f1a5-4334-be6f-992a43547ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:03.266777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74dcbf0-f1a5-4334-be6f-992a43547ab4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:03.280176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74dcbf0-f1a5-4334-be6f-992a43547ab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:03.282011 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74dcbf0-f1a5-4334-be6f-992a43547ab4 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:03.290358 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a1863-2861-4cc4-8030-128644a36314 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:03.303177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-f74dcbf0-f1a5-4334-be6f-992a43547ab4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:03.303177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:03.303177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:03.303177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:03.303177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:03.303177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:03.303177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:03.303177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:03.303177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:03.303177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:03.303177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:03.303177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:03.303177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:03.303177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:03.303177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:03.304153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:03.304153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:03.304153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:03.304153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:03.304153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:03.304153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:03.304153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:03.304153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:03.304153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:03.304153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:03.304153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:03.304153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:03.304153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:03.304153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:03.304153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:03.304153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:03.304153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:03.305023 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:03.305023 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:03.305023 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:03.305023 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:03.311726 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 328/655] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:03.311726 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74dcbf0-f1a5-4334-be6f-992a43547ab4 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:03.335012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86b0b0-33c1-4f81-a6fc-f9011860a666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:03.335686 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86b0b0-33c1-4f81-a6fc-f9011860a666 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:03.335993 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86b0b0-33c1-4f81-a6fc-f9011860a666 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:03.336265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86b0b0-33c1-4f81-a6fc-f9011860a666 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:03.337506 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86b0b0-33c1-4f81-a6fc-f9011860a666 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:03.337841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86b0b0-33c1-4f81-a6fc-f9011860a666 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:03.339584 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86b0b0-33c1-4f81-a6fc-f9011860a666 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:03.367460 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 328/656] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Jun 6 10:25:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:03.367460 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86b0b0-33c1-4f81-a6fc-f9011860a666 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:03.435123 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74dcbf0-f1a5-4334-be6f-992a43547ab4 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:03.468888 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da278f-979f-492d-a176-32939f8b6dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:03.469678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da278f-979f-492d-a176-32939f8b6dce None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:03.469956 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da278f-979f-492d-a176-32939f8b6dce None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:03.470229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da278f-979f-492d-a176-32939f8b6dce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:03.472542 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:03.481748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da278f-979f-492d-a176-32939f8b6dce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:03.483168 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da278f-979f-492d-a176-32939f8b6dce None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:03.486463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86b0b0-33c1-4f81-a6fc-f9011860a666 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:03.500660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-27da278f-979f-492d-a176-32939f8b6dce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:03.500660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:03.500660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:03.500660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:03.500660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:03.500660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:03.500660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:03.500660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:03.500660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:03.500660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:03.500660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:03.500660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:03.500660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:03.500660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:03.500660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:03.501580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:03.501580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:03.501580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:03.501580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:03.501580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:03.501580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:03.501580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:03.501580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:03.501580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:03.501580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:03.501580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:03.501580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:03.501580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:03.501580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:03.501580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:03.501580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:03.501580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:03.502447 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:03.502447 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:03.502447 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:03.502447 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:03.502447 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 329/657] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:03.503246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da278f-979f-492d-a176-32939f8b6dce None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:03.525122 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6b019-44c1-4f9b-bad5-39cfa1e9af00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:03.525890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6b019-44c1-4f9b-bad5-39cfa1e9af00 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:03.526168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6b019-44c1-4f9b-bad5-39cfa1e9af00 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:03.526439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6b019-44c1-4f9b-bad5-39cfa1e9af00 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:03.527261 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b6b019-44c1-4f9b-bad5-39cfa1e9af00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:03.527566 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b6b019-44c1-4f9b-bad5-39cfa1e9af00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:03.529077 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b6b019-44c1-4f9b-bad5-39cfa1e9af00 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:03.542822 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:03.546258 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 329/658] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:03.547641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13416417-1ea8-4002-bcbd-522cec68025b None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:03.547955 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13416417-1ea8-4002-bcbd-522cec68025b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:03.548219 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13416417-1ea8-4002-bcbd-522cec68025b None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:03.551826 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 330/659] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:03.551826 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13416417-1ea8-4002-bcbd-522cec68025b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:03.617914 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da278f-979f-492d-a176-32939f8b6dce None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:03.654992 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a22e5-a34b-4dc6-b3ef-fd4ba40fb50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:03.655632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a22e5-a34b-4dc6-b3ef-fd4ba40fb50b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:03.655966 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a22e5-a34b-4dc6-b3ef-fd4ba40fb50b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:03.656212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a22e5-a34b-4dc6-b3ef-fd4ba40fb50b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:03.659009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13416417-1ea8-4002-bcbd-522cec68025b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:03.694277 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a28751b-dada-4dd4-b898-c556fc1b4bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:03.695296 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28751b-dada-4dd4-b898-c556fc1b4bac None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:03.695579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28751b-dada-4dd4-b898-c556fc1b4bac None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:03.695886 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28751b-dada-4dd4-b898-c556fc1b4bac None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:03.696611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a28751b-dada-4dd4-b898-c556fc1b4bac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:03.696916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a28751b-dada-4dd4-b898-c556fc1b4bac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:03.698317 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a28751b-dada-4dd4-b898-c556fc1b4bac None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:03.716048 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 331/660] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:03.716048 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a28751b-dada-4dd4-b898-c556fc1b4bac None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:03.725739 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:03.735461 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9a22e5-a34b-4dc6-b3ef-fd4ba40fb50b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:03.736982 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9a22e5-a34b-4dc6-b3ef-fd4ba40fb50b None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:03.774413 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:03.820501 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a28751b-dada-4dd4-b898-c556fc1b4bac None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:03.851113 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 330/661] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:25:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:25:03.852724 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a62c7e-1075-4848-b346-a39e0a2daa6f None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:03.853011 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a62c7e-1075-4848-b346-a39e0a2daa6f None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:03.853287 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a62c7e-1075-4848-b346-a39e0a2daa6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:03.860294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2cba9-26d3-4830-95e4-a61b13862bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:03.861372 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2cba9-26d3-4830-95e4-a61b13862bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:03.861847 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2cba9-26d3-4830-95e4-a61b13862bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:03.862825 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2cba9-26d3-4830-95e4-a61b13862bc4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:03.870108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-86a62c7e-1075-4848-b346-a39e0a2daa6f None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:03.871247 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f2cba9-26d3-4830-95e4-a61b13862bc4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:03.871383 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-86a62c7e-1075-4848-b346-a39e0a2daa6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:03.873370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f2cba9-26d3-4830-95e4-a61b13862bc4 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:03.893185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-34f2cba9-26d3-4830-95e4-a61b13862bc4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 10:25:03.893185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:03.893185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:03.893185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:03.893185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:03.893185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:03.893185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:03.893185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:03.893185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:03.893185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:03.893185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:03.893185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:03.893185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:25:03.893185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:25:03.893185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:25:03.893185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:25:03.894561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:03.894561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:03.894561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:03.894561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:03.894561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:03.894561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:03.894561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:03.894561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:03.894561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:03.894561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:03.894561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:03.894561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:03.894561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:03.894561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:03.894561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:03.894561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:03.894561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:03.894561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:03.895639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:03.895639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:03.895639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:25:03.895639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:25:03.895639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:25:03.895639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:25:03.895639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:25:03.895639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:25:03.895639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 10:25:03.895639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:03.897875 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 332/662] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:25:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:03.898963 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2cba9-26d3-4830-95e4-a61b13862bc4 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:03.945065 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 331/663] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:25:03.946183 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a62c7e-1075-4848-b346-a39e0a2daa6f None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:03.988787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:04.017062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:04.025152 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2cba9-26d3-4830-95e4-a61b13862bc4 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:04.060669 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a62c7e-1075-4848-b346-a39e0a2daa6f None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:04.060931 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9053a-8643-4aee-b667-7535949f985b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:04.061662 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9053a-8643-4aee-b667-7535949f985b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:04.061933 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9053a-8643-4aee-b667-7535949f985b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:04.062213 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9053a-8643-4aee-b667-7535949f985b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:04.070569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9053a-8643-4aee-b667-7535949f985b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:04.072034 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9053a-8643-4aee-b667-7535949f985b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:04.086033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-05c9053a-8643-4aee-b667-7535949f985b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:04.086033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:04.086033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:04.086033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:04.086033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:04.086033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:04.086033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:04.086033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:04.086033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:04.086033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:04.086033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:04.086033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:04.086033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:04.086033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:04.086033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:04.086953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:04.086953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:04.086953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:04.086953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:04.086953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:04.086953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:04.086953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:04.086953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:04.086953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:04.086953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:04.086953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:04.086953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:04.086953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:04.086953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:04.086953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:04.086953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:04.086953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:04.088393 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:04.088393 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:04.088393 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:04.088393 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:04.088393 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:04.088393 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:04.088393 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:04.088393 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:04.088393 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 333/664] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:04.088393 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9053a-8643-4aee-b667-7535949f985b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:04.101439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440421b-f0e9-45bb-9080-09ffbc57618f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:04.102238 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440421b-f0e9-45bb-9080-09ffbc57618f None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:04.102521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440421b-f0e9-45bb-9080-09ffbc57618f None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:04.102800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440421b-f0e9-45bb-9080-09ffbc57618f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:04.114086 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440421b-f0e9-45bb-9080-09ffbc57618f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:04.115516 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440421b-f0e9-45bb-9080-09ffbc57618f None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:04.134020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-c440421b-f0e9-45bb-9080-09ffbc57618f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:04.134020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:04.134020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:04.134020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:04.134020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:04.134020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:04.134020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:04.134020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:04.134020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:04.134020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:04.134020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:04.134020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:04.134020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:04.134020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:04.134020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:04.139521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:04.139521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:04.139521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:04.139521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:04.139521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:04.139521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:04.139521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:04.139521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:04.139521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:04.139521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:04.139521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:04.139521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:04.139521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:04.139521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:04.139521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:04.139521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:04.139521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:04.140694 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:04.140694 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:04.140694 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:04.140694 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:04.140694 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 332/665] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:04.140694 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440421b-f0e9-45bb-9080-09ffbc57618f None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:04.200505 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9053a-8643-4aee-b667-7535949f985b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:04.226569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:04.234905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e32b90-3295-40b5-9d22-fdb0050a2b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:04.235586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e32b90-3295-40b5-9d22-fdb0050a2b69 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:04.235905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e32b90-3295-40b5-9d22-fdb0050a2b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:04.236184 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e32b90-3295-40b5-9d22-fdb0050a2b69 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:04.237021 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e32b90-3295-40b5-9d22-fdb0050a2b69 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:04.237438 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e32b90-3295-40b5-9d22-fdb0050a2b69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:04.239362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e32b90-3295-40b5-9d22-fdb0050a2b69 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:04.249598 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440421b-f0e9-45bb-9080-09ffbc57618f None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:04.256562 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 334/666] 10.222.0.42 () {56 vars in 1139 bytes} [Mon Jun 6 10:25:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:04.257384 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e32b90-3295-40b5-9d22-fdb0050a2b69 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:04.274990 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:04.296440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfdc316-2df0-4e98-adb9-924c4d3d0793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:04.297654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdc316-2df0-4e98-adb9-924c4d3d0793 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:04.298111 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdc316-2df0-4e98-adb9-924c4d3d0793 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:04.298628 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdc316-2df0-4e98-adb9-924c4d3d0793 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:04.310663 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfdc316-2df0-4e98-adb9-924c4d3d0793 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:25:04.319953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-2bfdc316-2df0-4e98-adb9-924c4d3d0793 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:25:04.319953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:04.319953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:04.319953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:04.319953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:04.319953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:04.319953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:04.319953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:04.319953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:04.319953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:04.319953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:04.319953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:04.319953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:25:04.319953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:25:04.319953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:25:04.319953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:25:04.320892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:25:04.320892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:25:04.320892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:25:04.320892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:25:04.320892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:04.320892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:04.320892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:04.320892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:04.320892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:04.320892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:04.320892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:04.320892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:04.320892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:04.320892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:04.320892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:04.320892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:04.326578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:04.326578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:04.326578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:04.326578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:04.326578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:25:04.326578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:25:04.326578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:25:04.326578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:25:04.326578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:25:04.326578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:25:04.326578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:25:04.326578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:04.326578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 333/667] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:25:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:04.326578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfdc316-2df0-4e98-adb9-924c4d3d0793 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:04.387477 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e32b90-3295-40b5-9d22-fdb0050a2b69 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:04.425924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127f85da-6880-4872-a3cc-588e706c66a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:04.426843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f85da-6880-4872-a3cc-588e706c66a3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:04.426956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f85da-6880-4872-a3cc-588e706c66a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:04.427227 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f85da-6880-4872-a3cc-588e706c66a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:04.428042 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127f85da-6880-4872-a3cc-588e706c66a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:04.428346 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127f85da-6880-4872-a3cc-588e706c66a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:04.429781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127f85da-6880-4872-a3cc-588e706c66a3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:04.439677 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfdc316-2df0-4e98-adb9-924c4d3d0793 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:04.443361 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 335/668] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:25:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:04.443903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127f85da-6880-4872-a3cc-588e706c66a3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:04.479671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a6211-c039-4577-a039-8e7a266e5af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:04.481048 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a6211-c039-4577-a039-8e7a266e5af4 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:04.481335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a6211-c039-4577-a039-8e7a266e5af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:04.481609 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a6211-c039-4577-a039-8e7a266e5af4 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:04.482333 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a6211-c039-4577-a039-8e7a266e5af4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:04.482670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a6211-c039-4577-a039-8e7a266e5af4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:04.484037 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a6211-c039-4577-a039-8e7a266e5af4 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:04.492518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:04.498452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 334/669] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:04.499235 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a6211-c039-4577-a039-8e7a266e5af4 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:04.543771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:04.559674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127f85da-6880-4872-a3cc-588e706c66a3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:04.605647 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69c805-161b-4c81-b864-9820a7be8f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:04.606295 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69c805-161b-4c81-b864-9820a7be8f6f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:04.606573 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69c805-161b-4c81-b864-9820a7be8f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:04.606845 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69c805-161b-4c81-b864-9820a7be8f6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:04.610013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a6211-c039-4577-a039-8e7a266e5af4 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:04.665947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3f3b5-9a2c-42b3-ad4c-9ecdbfd13c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:04.666795 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3f3b5-9a2c-42b3-ad4c-9ecdbfd13c74 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:04.667142 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3f3b5-9a2c-42b3-ad4c-9ecdbfd13c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:04.667487 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3f3b5-9a2c-42b3-ad4c-9ecdbfd13c74 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:04.668315 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3f3b5-9a2c-42b3-ad4c-9ecdbfd13c74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:04.668619 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3f3b5-9a2c-42b3-ad4c-9ecdbfd13c74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:04.670539 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3f3b5-9a2c-42b3-ad4c-9ecdbfd13c74 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:04.696033 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 335/670] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:25:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:04.696033 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3f3b5-9a2c-42b3-ad4c-9ecdbfd13c74 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:04.713684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69c805-161b-4c81-b864-9820a7be8f6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:04.715133 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69c805-161b-4c81-b864-9820a7be8f6f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:04.773833 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:04.818795 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:04.822243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3f3b5-9a2c-42b3-ad4c-9ecdbfd13c74 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:04.832764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 336/671] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:25:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:25:04.833734 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69c805-161b-4c81-b864-9820a7be8f6f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:04.861307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3063fa0-3a6b-4905-a546-32219901c990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:04.861967 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3063fa0-3a6b-4905-a546-32219901c990 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:04.862285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3063fa0-3a6b-4905-a546-32219901c990 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:04.862680 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3063fa0-3a6b-4905-a546-32219901c990 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:04.864245 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3063fa0-3a6b-4905-a546-32219901c990 None admin] RBAC: Adding query filter params (role.id=95eccd30e39b4105ac6e8c0d9f973951, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=2b8836193f1a47e1a62ea5167e01b26e) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:04.864785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3063fa0-3a6b-4905-a546-32219901c990 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:04.866702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3063fa0-3a6b-4905-a546-32219901c990 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:04.884055 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 336/672] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:25:04 2022] GET /identity/v3/role_assignments?user.id=2b8836193f1a47e1a62ea5167e01b26e&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=95eccd30e39b4105ac6e8c0d9f973951 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:04.884055 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3063fa0-3a6b-4905-a546-32219901c990 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:04.962480 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69c805-161b-4c81-b864-9820a7be8f6f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:05.009770 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6018a65-748d-4178-8bfd-fe7029e903a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:05.010587 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6018a65-748d-4178-8bfd-fe7029e903a4 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:05.010873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6018a65-748d-4178-8bfd-fe7029e903a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:05.011148 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6018a65-748d-4178-8bfd-fe7029e903a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:05.011247 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3063fa0-3a6b-4905-a546-32219901c990 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:05.024945 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6018a65-748d-4178-8bfd-fe7029e903a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:05.026355 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6018a65-748d-4178-8bfd-fe7029e903a4 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:05.041969 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-a6018a65-748d-4178-8bfd-fe7029e903a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:05.041969 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:05.041969 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:05.041969 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:05.041969 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:05.041969 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:05.041969 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:05.041969 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:05.041969 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:05.041969 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:05.041969 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:05.041969 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:05.041969 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:05.041969 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:05.041969 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:05.043299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:05.043299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:05.043299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:05.043299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:05.043299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:05.043299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:05.043299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:05.043299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:05.043299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:05.043299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:05.043299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:05.043299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:05.043299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:05.043299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:05.043299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:05.043299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:05.043299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:05.044464 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:05.044464 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:05.044464 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:05.044464 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:05.044464 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:05.044464 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:05.044464 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:05.044464 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:05.044464 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 337/673] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:05.044464 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6018a65-748d-4178-8bfd-fe7029e903a4 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:05.061175 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:05.063306 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361bb088-fceb-460b-8741-450b5f0664f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:05.064188 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361bb088-fceb-460b-8741-450b5f0664f1 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:05.064477 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361bb088-fceb-460b-8741-450b5f0664f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:05.064757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361bb088-fceb-460b-8741-450b5f0664f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:05.077103 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361bb088-fceb-460b-8741-450b5f0664f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:05.078563 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361bb088-fceb-460b-8741-450b5f0664f1 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:05.099760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-361bb088-fceb-460b-8741-450b5f0664f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:05.099760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:05.099760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:05.099760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:05.099760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:05.099760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:05.099760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:05.099760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:05.099760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:05.099760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:05.099760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:05.099760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:05.099760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:05.099760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:05.099760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:05.100941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:05.100941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:05.100941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:05.100941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:05.100941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:05.100941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:05.100941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:05.100941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:05.100941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:05.100941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:05.100941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:05.100941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:05.100941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:05.100941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:05.100941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:05.100941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:05.100941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:05.102008 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:05.102008 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:05.102008 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:05.102008 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:05.102008 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 337/674] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:05.102282 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaad766-2be7-4550-be50-705ab4adb0eb None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:05.102664 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaad766-2be7-4550-be50-705ab4adb0eb None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:05.103835 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaad766-2be7-4550-be50-705ab4adb0eb None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:05.105195 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 338/675] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:05.105195 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaad766-2be7-4550-be50-705ab4adb0eb None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:05.125193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:05.178486 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6018a65-748d-4178-8bfd-fe7029e903a4 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:05.213881 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c52c2-db6a-405f-a7f1-e96188aed6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:05.214676 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c52c2-db6a-405f-a7f1-e96188aed6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:05.214963 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c52c2-db6a-405f-a7f1-e96188aed6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:05.215223 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c52c2-db6a-405f-a7f1-e96188aed6a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:05.221000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaad766-2be7-4550-be50-705ab4adb0eb None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:05.226064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c52c2-db6a-405f-a7f1-e96188aed6a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:05.227517 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c52c2-db6a-405f-a7f1-e96188aed6a5 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:05.245786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-185c52c2-db6a-405f-a7f1-e96188aed6a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:05.245786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:05.245786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:05.245786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:05.245786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:05.245786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:05.245786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:05.245786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:05.245786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:05.245786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:05.245786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:05.245786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:05.245786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:05.245786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:05.245786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:05.246670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:05.246670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:05.246670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:05.246670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:05.246670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:05.246670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:05.246670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:05.246670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:05.246670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:05.246670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:05.246670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:05.246670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:05.246670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:05.246670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:05.246670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:05.246670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:05.246670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:05.247563 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:05.247563 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:05.247563 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:05.247563 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:05.251360 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 338/676] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:05.252209 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c52c2-db6a-405f-a7f1-e96188aed6a5 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:05.263506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35bd44-2c45-4463-82a3-0de1005112b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:05.264542 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35bd44-2c45-4463-82a3-0de1005112b6 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:05.264827 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35bd44-2c45-4463-82a3-0de1005112b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:05.265427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35bd44-2c45-4463-82a3-0de1005112b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:05.266220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c35bd44-2c45-4463-82a3-0de1005112b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:05.266532 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c35bd44-2c45-4463-82a3-0de1005112b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:05.267988 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c35bd44-2c45-4463-82a3-0de1005112b6 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:05.282693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 339/677] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:25:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:05.283695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71f5ac-6cd3-40f5-bd41-e17f092fdb1d None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:05.284017 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71f5ac-6cd3-40f5-bd41-e17f092fdb1d None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:05.284332 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71f5ac-6cd3-40f5-bd41-e17f092fdb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:05.301138 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-ea71f5ac-6cd3-40f5-bd41-e17f092fdb1d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:05.302345 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ea71f5ac-6cd3-40f5-bd41-e17f092fdb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:05.321840 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:05.366481 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c52c2-db6a-405f-a7f1-e96188aed6a5 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:05.385719 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 340/678] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:25:05.386040 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71f5ac-6cd3-40f5-bd41-e17f092fdb1d None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:05.395206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:05.419728 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c8cec-05fd-4c58-92bd-8ed287180ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:05.420459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c8cec-05fd-4c58-92bd-8ed287180ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:05.420764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c8cec-05fd-4c58-92bd-8ed287180ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:05.421090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c8cec-05fd-4c58-92bd-8ed287180ec6 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:05.421718 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c8cec-05fd-4c58-92bd-8ed287180ec6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:05.422022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c8cec-05fd-4c58-92bd-8ed287180ec6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:05.423386 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c8cec-05fd-4c58-92bd-8ed287180ec6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:05.437431 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 339/679] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:05.438399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c8cec-05fd-4c58-92bd-8ed287180ec6 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:05.495818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71f5ac-6cd3-40f5-bd41-e17f092fdb1d None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:05.531804 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be1c97-87f6-4906-ad44-75cc251ca3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:05.532491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be1c97-87f6-4906-ad44-75cc251ca3bd None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:05.532823 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be1c97-87f6-4906-ad44-75cc251ca3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:05.533047 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be1c97-87f6-4906-ad44-75cc251ca3bd None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:05.533765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be1c97-87f6-4906-ad44-75cc251ca3bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:05.534072 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be1c97-87f6-4906-ad44-75cc251ca3bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:05.535459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be1c97-87f6-4906-ad44-75cc251ca3bd None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:05.551311 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 341/680] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:05.551311 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be1c97-87f6-4906-ad44-75cc251ca3bd None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:05.554008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c8cec-05fd-4c58-92bd-8ed287180ec6 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:05.572681 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:05.593357 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4932b5-baab-4213-8a84-99894764a109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:05.594029 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4932b5-baab-4213-8a84-99894764a109 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:05.594299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4932b5-baab-4213-8a84-99894764a109 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:05.594572 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4932b5-baab-4213-8a84-99894764a109 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:05.595402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4932b5-baab-4213-8a84-99894764a109 None admin] RBAC: Adding query filter params (role.id=95eccd30e39b4105ac6e8c0d9f973951, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=1a67e007fc314b34aac2471a7c20f240) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:05.595721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4932b5-baab-4213-8a84-99894764a109 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:05.597310 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4932b5-baab-4213-8a84-99894764a109 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:05.612279 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 340/681] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:25:05 2022] GET /identity/v3/role_assignments?user.id=1a67e007fc314b34aac2471a7c20f240&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=95eccd30e39b4105ac6e8c0d9f973951 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:05.613528 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4932b5-baab-4213-8a84-99894764a109 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:05.625869 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:05.672473 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be1c97-87f6-4906-ad44-75cc251ca3bd None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:05.715440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20682a1b-06ef-4866-9b17-8d6b893b6e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:05.716121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20682a1b-06ef-4866-9b17-8d6b893b6e6d None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:05.716408 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20682a1b-06ef-4866-9b17-8d6b893b6e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:05.716688 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20682a1b-06ef-4866-9b17-8d6b893b6e6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:05.729763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20682a1b-06ef-4866-9b17-8d6b893b6e6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:25:05.733840 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4932b5-baab-4213-8a84-99894764a109 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:05.738321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-20682a1b-06ef-4866-9b17-8d6b893b6e6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:25:05.738321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:05.738321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:05.738321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:05.738321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:05.738321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:05.738321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:05.738321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:05.738321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:05.738321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:05.738321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:05.738321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:05.738321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:25:05.738321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:25:05.738321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:25:05.738321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:25:05.739307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:25:05.739307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:25:05.739307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:25:05.739307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:25:05.739307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:05.739307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:05.739307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:05.739307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:05.739307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:05.739307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:05.739307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:05.739307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:05.739307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:05.739307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:05.739307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:05.739307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:05.740182 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:05.740182 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:05.740182 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:05.740182 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:05.740182 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:25:05.740182 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:25:05.740182 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:25:05.740182 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:25:05.740182 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:25:05.740182 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:25:05.740182 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:25:05.740182 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:05.744270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 342/682] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:25:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:05.745882 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20682a1b-06ef-4866-9b17-8d6b893b6e6d None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:05.777221 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda33321-904f-4902-9852-733d8ec437a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:05.778416 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda33321-904f-4902-9852-733d8ec437a5 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:05.778905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda33321-904f-4902-9852-733d8ec437a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:05.779390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda33321-904f-4902-9852-733d8ec437a5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:05.789042 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda33321-904f-4902-9852-733d8ec437a5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:05.790955 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda33321-904f-4902-9852-733d8ec437a5 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:05.807292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-dda33321-904f-4902-9852-733d8ec437a5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 10:25:05.807292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:05.807292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:05.807292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:05.807292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:05.807292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:05.807292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:05.807292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:05.807292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:05.807292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:05.807292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:05.807292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:05.807292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:25:05.807292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:25:05.807292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:25:05.807292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:25:05.808300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:05.808300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:05.808300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:05.808300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:05.808300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:05.808300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:05.808300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:05.808300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:05.808300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:05.808300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:05.808300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:05.808300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:05.808300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:05.808300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:05.808300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:05.808300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:05.808300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:05.808300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:05.809255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:05.809255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:05.809255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:25:05.809255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:25:05.809255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:25:05.809255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:25:05.809255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:25:05.809255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:25:05.809255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 10:25:05.809255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:05.811569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 341/683] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jun 6 10:25:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:05.812641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda33321-904f-4902-9852-733d8ec437a5 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:05.862723 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:05.865328 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20682a1b-06ef-4866-9b17-8d6b893b6e6d None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:05.899908 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:05.916283 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737ebd7-e06f-4f3f-9239-ae7ebe07a20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:05.917210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737ebd7-e06f-4f3f-9239-ae7ebe07a20c None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:05.917494 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737ebd7-e06f-4f3f-9239-ae7ebe07a20c None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:05.917772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737ebd7-e06f-4f3f-9239-ae7ebe07a20c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:05.930686 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737ebd7-e06f-4f3f-9239-ae7ebe07a20c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:05.932265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737ebd7-e06f-4f3f-9239-ae7ebe07a20c None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:05.935978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda33321-904f-4902-9852-733d8ec437a5 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:05.946695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-4737ebd7-e06f-4f3f-9239-ae7ebe07a20c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:05.946695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:05.946695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:05.946695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:05.946695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:05.946695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:05.946695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:05.946695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:05.946695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:05.946695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:05.946695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:05.946695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:05.946695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:05.946695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:05.946695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:05.947603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:05.947603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:05.947603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:05.947603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:05.947603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:05.947603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:05.947603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:05.947603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:05.947603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:05.947603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:05.947603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:05.947603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:05.947603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:05.947603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:05.947603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:05.947603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:05.947603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:05.949553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:05.949553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:05.949553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:05.949553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:05.949553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:05.949553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:05.949553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:05.949553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:05.949553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 343/684] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:05.949553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737ebd7-e06f-4f3f-9239-ae7ebe07a20c None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:05.979391 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58abab8-a85c-4c0e-9804-debc6c9921b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:05.980192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58abab8-a85c-4c0e-9804-debc6c9921b2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:05.980481 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58abab8-a85c-4c0e-9804-debc6c9921b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:05.980757 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58abab8-a85c-4c0e-9804-debc6c9921b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:05.981499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58abab8-a85c-4c0e-9804-debc6c9921b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:05.981869 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58abab8-a85c-4c0e-9804-debc6c9921b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:05.983460 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58abab8-a85c-4c0e-9804-debc6c9921b2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:05.992530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 342/685] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jun 6 10:25:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:05.993418 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58abab8-a85c-4c0e-9804-debc6c9921b2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:06.073116 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737ebd7-e06f-4f3f-9239-ae7ebe07a20c None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:06.113305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58abab8-a85c-4c0e-9804-debc6c9921b2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:06.114414 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082038ad-1bf1-4a2c-90a9-85e42c6f5f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:06.115160 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082038ad-1bf1-4a2c-90a9-85e42c6f5f19 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:06.115544 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082038ad-1bf1-4a2c-90a9-85e42c6f5f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:06.115984 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082038ad-1bf1-4a2c-90a9-85e42c6f5f19 None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:06.117591 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:06.117709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082038ad-1bf1-4a2c-90a9-85e42c6f5f19 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:06.117850 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082038ad-1bf1-4a2c-90a9-85e42c6f5f19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:06.119417 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082038ad-1bf1-4a2c-90a9-85e42c6f5f19 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:06.141318 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 344/686] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jun 6 10:25:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:06.142604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082038ad-1bf1-4a2c-90a9-85e42c6f5f19 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:06.156779 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab5d83-d2bf-4d3e-b926-ea397ca4f234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:06.157909 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab5d83-d2bf-4d3e-b926-ea397ca4f234 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:06.158268 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab5d83-d2bf-4d3e-b926-ea397ca4f234 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:06.158625 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab5d83-d2bf-4d3e-b926-ea397ca4f234 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:06.159556 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab5d83-d2bf-4d3e-b926-ea397ca4f234 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:06.160260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab5d83-d2bf-4d3e-b926-ea397ca4f234 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:06.162097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab5d83-d2bf-4d3e-b926-ea397ca4f234 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:06.179862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 343/687] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:25:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:06.180986 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab5d83-d2bf-4d3e-b926-ea397ca4f234 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:06.209903 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:06.274369 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082038ad-1bf1-4a2c-90a9-85e42c6f5f19 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:06.289210 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab5d83-d2bf-4d3e-b926-ea397ca4f234 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:06.321082 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae9105-6718-4b0e-8494-829eaab28708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:06.323197 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae9105-6718-4b0e-8494-829eaab28708 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:06.323640 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae9105-6718-4b0e-8494-829eaab28708 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:06.324107 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae9105-6718-4b0e-8494-829eaab28708 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:06.327132 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf18d9-77eb-40bb-814b-539c4fc26c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:06.327881 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf18d9-77eb-40bb-814b-539c4fc26c23 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:06.328238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf18d9-77eb-40bb-814b-539c4fc26c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:06.328581 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf18d9-77eb-40bb-814b-539c4fc26c23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:06.340131 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae9105-6718-4b0e-8494-829eaab28708 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:06.342705 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae9105-6718-4b0e-8494-829eaab28708 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:06.363819 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:06.369922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-f3ae9105-6718-4b0e-8494-829eaab28708 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:06.369922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:06.369922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:06.369922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:06.369922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:06.369922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:06.369922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:06.369922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:06.369922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:06.369922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:06.369922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:06.369922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:06.369922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:06.369922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:06.369922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:06.370825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:06.370825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:06.370825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:06.370825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:06.370825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:06.370825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:06.370825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:06.370825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:06.370825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:06.370825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:06.370825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:06.370825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:06.370825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:06.370825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:06.370825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:06.370825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:06.370825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:06.371713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:06.371713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:06.371713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:06.371713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:06.379115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 345/688] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:06.379115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae9105-6718-4b0e-8494-829eaab28708 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:06.416072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf18d9-77eb-40bb-814b-539c4fc26c23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:06.417918 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf18d9-77eb-40bb-814b-539c4fc26c23 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:06.496954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:06.505485 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae9105-6718-4b0e-8494-829eaab28708 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:06.527206 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 344/689] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:25:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:25:06.528359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf18d9-77eb-40bb-814b-539c4fc26c23 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:06.553788 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c48d0e-f087-4bc8-9779-81188cee0f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:06.554444 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c48d0e-f087-4bc8-9779-81188cee0f2f None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:06.554725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c48d0e-f087-4bc8-9779-81188cee0f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:06.554999 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c48d0e-f087-4bc8-9779-81188cee0f2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:06.555879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c48d0e-f087-4bc8-9779-81188cee0f2f None admin] RBAC: Adding query filter params (role.id=95eccd30e39b4105ac6e8c0d9f973951, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=881069f9b40a485dac2dda684dd6cd3b) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:06.556259 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c48d0e-f087-4bc8-9779-81188cee0f2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:06.557733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c48d0e-f087-4bc8-9779-81188cee0f2f None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:06.568167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 346/690] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:25:06 2022] GET /identity/v3/role_assignments?user.id=881069f9b40a485dac2dda684dd6cd3b&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=95eccd30e39b4105ac6e8c0d9f973951 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:06.568167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c48d0e-f087-4bc8-9779-81188cee0f2f None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:06.639295 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:06.717921 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf18d9-77eb-40bb-814b-539c4fc26c23 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:06.750174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c48d0e-f087-4bc8-9779-81188cee0f2f None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:06.786200 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef87b9-b587-420b-adde-658bd1477b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:06.787269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef87b9-b587-420b-adde-658bd1477b57 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:06.787774 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef87b9-b587-420b-adde-658bd1477b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:06.788356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef87b9-b587-420b-adde-658bd1477b57 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:06.789166 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef87b9-b587-420b-adde-658bd1477b57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:06.789549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef87b9-b587-420b-adde-658bd1477b57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:06.790990 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef87b9-b587-420b-adde-658bd1477b57 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:06.810113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 345/691] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:06.812086 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc0ba5-400e-47fa-a005-3e8a983378b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:06.813108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc0ba5-400e-47fa-a005-3e8a983378b3 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:06.813567 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc0ba5-400e-47fa-a005-3e8a983378b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:06.814058 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc0ba5-400e-47fa-a005-3e8a983378b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:06.816854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef87b9-b587-420b-adde-658bd1477b57 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:06.826561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc0ba5-400e-47fa-a005-3e8a983378b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:06.828465 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc0ba5-400e-47fa-a005-3e8a983378b3 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:06.849125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-3edc0ba5-400e-47fa-a005-3e8a983378b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:06.849125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:06.849125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:06.849125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:06.849125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:06.849125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:06.849125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:06.849125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:06.849125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:06.849125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:06.849125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:06.849125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:06.849125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:06.849125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:06.849125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:06.850170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:06.850170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:06.850170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:06.850170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:06.850170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:06.850170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:06.850170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:06.850170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:06.850170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:06.850170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:06.850170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:06.850170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:06.850170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:06.850170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:06.850170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:06.850170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:06.850170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:06.851062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:06.851062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:06.851062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:06.851062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:06.853172 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 347/692] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:06.857972 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc0ba5-400e-47fa-a005-3e8a983378b3 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:06.877948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:06.934812 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef87b9-b587-420b-adde-658bd1477b57 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:06.966490 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:06.980346 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee897a46-3112-43af-855b-7b8f54cf4160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:06.981155 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee897a46-3112-43af-855b-7b8f54cf4160 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:06.981437 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee897a46-3112-43af-855b-7b8f54cf4160 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:06.981712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee897a46-3112-43af-855b-7b8f54cf4160 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:06.991414 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee897a46-3112-43af-855b-7b8f54cf4160 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:06.992915 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee897a46-3112-43af-855b-7b8f54cf4160 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:06.993517 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc0ba5-400e-47fa-a005-3e8a983378b3 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:07.009787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-ee897a46-3112-43af-855b-7b8f54cf4160 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 10:25:07.009787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:07.009787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:07.009787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:07.009787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:07.009787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:07.009787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:07.009787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:07.009787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:07.009787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:07.009787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:07.009787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:07.009787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:25:07.009787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:25:07.009787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:25:07.009787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:25:07.010780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:07.010780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:07.010780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:07.010780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:07.010780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:07.010780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:07.010780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:07.010780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:07.010780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:07.010780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:07.010780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:07.010780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:07.010780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:07.010780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:07.010780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:07.010780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:07.010780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:07.010780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:07.011707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:07.011707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:07.011707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:25:07.011707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:25:07.011707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:25:07.011707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:25:07.011707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:25:07.011707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:25:07.011707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 10:25:07.011707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:07.011707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 346/693] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:25:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:07.018262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee897a46-3112-43af-855b-7b8f54cf4160 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:07.047159 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffccdd7-baf1-460c-a15c-becbcae7d91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:07.048441 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffccdd7-baf1-460c-a15c-becbcae7d91b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:07.048937 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffccdd7-baf1-460c-a15c-becbcae7d91b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:07.049420 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffccdd7-baf1-460c-a15c-becbcae7d91b None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:07.050590 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffccdd7-baf1-460c-a15c-becbcae7d91b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:07.051071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffccdd7-baf1-460c-a15c-becbcae7d91b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:07.052669 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffccdd7-baf1-460c-a15c-becbcae7d91b None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:07.067920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 348/694] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:07.072917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffccdd7-baf1-460c-a15c-becbcae7d91b None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:07.185411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee897a46-3112-43af-855b-7b8f54cf4160 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:07.205615 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:07.260657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03cd96-73f4-4c5e-bca8-d29aa5871bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:07.261887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03cd96-73f4-4c5e-bca8-d29aa5871bcb None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:07.262947 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03cd96-73f4-4c5e-bca8-d29aa5871bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:07.263587 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03cd96-73f4-4c5e-bca8-d29aa5871bcb None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:07.265621 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffccdd7-baf1-460c-a15c-becbcae7d91b None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:07.266696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b03cd96-73f4-4c5e-bca8-d29aa5871bcb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:07.269157 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b03cd96-73f4-4c5e-bca8-d29aa5871bcb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:07.271183 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b03cd96-73f4-4c5e-bca8-d29aa5871bcb None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:07.298371 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 347/695] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Jun 6 10:25:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:07.306545 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03cd96-73f4-4c5e-bca8-d29aa5871bcb None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:07.324432 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d808c45-6415-463a-a1a8-daa1eb51a5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:07.325333 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d808c45-6415-463a-a1a8-daa1eb51a5b0 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:07.325610 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d808c45-6415-463a-a1a8-daa1eb51a5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:07.325878 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d808c45-6415-463a-a1a8-daa1eb51a5b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:07.333871 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d808c45-6415-463a-a1a8-daa1eb51a5b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:07.335301 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d808c45-6415-463a-a1a8-daa1eb51a5b0 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:07.350973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-0d808c45-6415-463a-a1a8-daa1eb51a5b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:07.350973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:07.350973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:07.350973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:07.350973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:07.350973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:07.350973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:07.350973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:07.350973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:07.350973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:07.350973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:07.350973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:07.350973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:07.350973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:07.350973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:07.351922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:07.351922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:07.351922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:07.351922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:07.351922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:07.351922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:07.351922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:07.351922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:07.351922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:07.351922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:07.351922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:07.351922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:07.351922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:07.351922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:07.351922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:07.351922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:07.351922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:07.352811 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:07.352811 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:07.352811 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:07.352811 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:07.352811 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:07.352811 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:07.352811 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:07.352811 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:07.354478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 349/696] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:07.355845 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fdd15-e488-476b-bf43-0531202ffa02 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:07.356131 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fdd15-e488-476b-bf43-0531202ffa02 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:07.356400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fdd15-e488-476b-bf43-0531202ffa02 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:07.359006 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:07.360326 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 350/697] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:07.361349 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fdd15-e488-476b-bf43-0531202ffa02 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:07.437697 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03cd96-73f4-4c5e-bca8-d29aa5871bcb None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:07.479666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbeb92-5d00-40b7-9a3a-37828cd040d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:07.481221 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbeb92-5d00-40b7-9a3a-37828cd040d3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:07.481805 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbeb92-5d00-40b7-9a3a-37828cd040d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:07.482278 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbeb92-5d00-40b7-9a3a-37828cd040d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:07.488190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fdd15-e488-476b-bf43-0531202ffa02 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:07.520056 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:07.527484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35604479-aada-426f-8f61-e14e5dc1e66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:07.528179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35604479-aada-426f-8f61-e14e5dc1e66f None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:07.528463 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35604479-aada-426f-8f61-e14e5dc1e66f None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:07.528743 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35604479-aada-426f-8f61-e14e5dc1e66f None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:07.529657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35604479-aada-426f-8f61-e14e5dc1e66f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:07.529931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35604479-aada-426f-8f61-e14e5dc1e66f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:07.531511 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35604479-aada-426f-8f61-e14e5dc1e66f None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:07.548889 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 351/698] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:25:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:07.550757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb470426-0534-4a72-b863-275b1e60a267 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:07.551086 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb470426-0534-4a72-b863-275b1e60a267 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:07.551319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb470426-0534-4a72-b863-275b1e60a267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:07.572417 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbeb92-5d00-40b7-9a3a-37828cd040d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:07.572943 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-cb470426-0534-4a72-b863-275b1e60a267 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:07.574121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-cb470426-0534-4a72-b863-275b1e60a267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:07.574871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbeb92-5d00-40b7-9a3a-37828cd040d3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:07.631635 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:07.677965 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 352/699] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:25:07.678894 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb470426-0534-4a72-b863-275b1e60a267 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:07.709591 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 348/700] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:25:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 10:25:07.715103 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbeb92-5d00-40b7-9a3a-37828cd040d3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:07.815064 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb470426-0534-4a72-b863-275b1e60a267 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:07.824072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbeb92-5d00-40b7-9a3a-37828cd040d3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:07.840020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:07.853141 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f716a16-db6d-436d-8b2a-62e62f1102b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:07.854117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f716a16-db6d-436d-8b2a-62e62f1102b0 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:07.854595 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f716a16-db6d-436d-8b2a-62e62f1102b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:07.855169 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f716a16-db6d-436d-8b2a-62e62f1102b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:07.856353 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f716a16-db6d-436d-8b2a-62e62f1102b0 None admin] RBAC: Adding query filter params (role.id=0f929204eb7b4ea8859338ed2d3ff75b, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=ed8a40e4f1034a8e99d0eb505e448db9) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:07.856864 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f716a16-db6d-436d-8b2a-62e62f1102b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:07.858615 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f716a16-db6d-436d-8b2a-62e62f1102b0 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:07.866187 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 353/701] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:25:07 2022] GET /identity/v3/role_assignments?user.id=ed8a40e4f1034a8e99d0eb505e448db9&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=0f929204eb7b4ea8859338ed2d3ff75b => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:07.867386 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f716a16-db6d-436d-8b2a-62e62f1102b0 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:07.878268 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74889d-dd7c-4ec0-bb4c-6a435d9bc196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:07.879328 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74889d-dd7c-4ec0-bb4c-6a435d9bc196 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:07.879439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74889d-dd7c-4ec0-bb4c-6a435d9bc196 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:07.879650 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74889d-dd7c-4ec0-bb4c-6a435d9bc196 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:07.880357 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d74889d-dd7c-4ec0-bb4c-6a435d9bc196 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:07.881781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d74889d-dd7c-4ec0-bb4c-6a435d9bc196 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:07.913119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-4d74889d-dd7c-4ec0-bb4c-6a435d9bc196 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 10:25:07.913119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:07.913119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:07.913119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:07.913119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:07.913119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:07.913119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:07.913119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:07.913119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 10:25:07.913119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:07.913119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:07.913119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:07.913119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:07.913119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:07.913119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:07.913119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:07.914164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:07.914164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:07.914164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:07.914164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:07.914164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:07.914164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:07.914164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:07.914164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:07.914164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:07.914164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:07.914164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:07.914164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:07.914164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:07.914164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:07.914164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:07.914164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:07.915076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:07.915076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:07.915076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:07.915076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:07.915076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:07.915076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:07.915076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:07.915076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:07.915076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:07.915076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:07.915076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:07.915076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:07.915076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:07.915076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 10:25:07.915076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:07.915076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 349/702] 10.222.0.42 () {56 vars in 1129 bytes} [Mon Jun 6 10:25:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:07.920464 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74889d-dd7c-4ec0-bb4c-6a435d9bc196 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:07.935183 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:07.995043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f716a16-db6d-436d-8b2a-62e62f1102b0 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:08.036021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242f7c5-b5c4-4c5b-903e-a68c25f1ae8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:08.037330 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242f7c5-b5c4-4c5b-903e-a68c25f1ae8b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:08.037813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242f7c5-b5c4-4c5b-903e-a68c25f1ae8b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:08.038307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242f7c5-b5c4-4c5b-903e-a68c25f1ae8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:08.052298 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242f7c5-b5c4-4c5b-903e-a68c25f1ae8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:08.054337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242f7c5-b5c4-4c5b-903e-a68c25f1ae8b None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:08.062838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74889d-dd7c-4ec0-bb4c-6a435d9bc196 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:08.075109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-9242f7c5-b5c4-4c5b-903e-a68c25f1ae8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:08.075109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:08.075109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:08.075109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:08.075109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:08.075109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:08.075109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:08.075109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:08.075109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:08.075109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:08.075109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:08.075109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:08.075109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:08.075109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:08.075109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:08.076133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:08.076133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:08.076133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:08.076133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:08.076133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:08.076133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:08.076133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:08.076133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:08.076133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:08.076133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:08.076133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:08.076133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:08.076133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:08.076133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:08.076133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:08.076133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:08.076133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:08.077017 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:08.077017 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:08.077017 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:08.077017 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:08.079431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 354/703] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:08.085539 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242f7c5-b5c4-4c5b-903e-a68c25f1ae8b None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:08.101514 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc85602-aae9-402f-bff8-14131fc4ecc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:08.102208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc85602-aae9-402f-bff8-14131fc4ecc3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:08.102493 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc85602-aae9-402f-bff8-14131fc4ecc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:08.102771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc85602-aae9-402f-bff8-14131fc4ecc3 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:08.103511 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc85602-aae9-402f-bff8-14131fc4ecc3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:08.103880 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc85602-aae9-402f-bff8-14131fc4ecc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:08.105290 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc85602-aae9-402f-bff8-14131fc4ecc3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:08.110839 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 350/704] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Jun 6 10:25:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:08.117899 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc85602-aae9-402f-bff8-14131fc4ecc3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:08.144127 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:08.225216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242f7c5-b5c4-4c5b-903e-a68c25f1ae8b None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:08.236292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:08.285547 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc85602-aae9-402f-bff8-14131fc4ecc3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:08.290415 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d07b29-7e48-40df-ad45-7eed81361249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:08.291138 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d07b29-7e48-40df-ad45-7eed81361249 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:08.291813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d07b29-7e48-40df-ad45-7eed81361249 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:08.292109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d07b29-7e48-40df-ad45-7eed81361249 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:08.292847 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d07b29-7e48-40df-ad45-7eed81361249 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:08.293209 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d07b29-7e48-40df-ad45-7eed81361249 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:08.295419 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d07b29-7e48-40df-ad45-7eed81361249 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:08.313566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 355/705] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:08.316242 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31674de-ba0b-4466-b18e-b88e4d2da25d None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:08.316512 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31674de-ba0b-4466-b18e-b88e4d2da25d None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:08.316775 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31674de-ba0b-4466-b18e-b88e4d2da25d None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:08.323028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 356/706] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:08.323975 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31674de-ba0b-4466-b18e-b88e4d2da25d None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:08.331381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4e168-f6b4-461f-b6dd-d46a17ffa8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:08.332457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4e168-f6b4-461f-b6dd-d46a17ffa8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:08.332944 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4e168-f6b4-461f-b6dd-d46a17ffa8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:08.333419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4e168-f6b4-461f-b6dd-d46a17ffa8c0 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:08.334395 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4e168-f6b4-461f-b6dd-d46a17ffa8c0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:08.334950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4e168-f6b4-461f-b6dd-d46a17ffa8c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:08.336666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4e168-f6b4-461f-b6dd-d46a17ffa8c0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:08.343224 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 351/707] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Jun 6 10:25:08 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:08.344886 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae562e9d-45c7-4ae7-879e-fe10a8b496c6 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:08.345406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae562e9d-45c7-4ae7-879e-fe10a8b496c6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:08.345897 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae562e9d-45c7-4ae7-879e-fe10a8b496c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:08.368106 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ae562e9d-45c7-4ae7-879e-fe10a8b496c6 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:08.369651 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ae562e9d-45c7-4ae7-879e-fe10a8b496c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:08.458488 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 352/708] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:25:08.486612 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae562e9d-45c7-4ae7-879e-fe10a8b496c6 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:08.566343 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:08.575958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31674de-ba0b-4466-b18e-b88e4d2da25d None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:08.592694 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:08.623258 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf3bf8-dd75-4ee1-b7f1-b2d623d56111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:08.624091 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf3bf8-dd75-4ee1-b7f1-b2d623d56111 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:08.624879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf3bf8-dd75-4ee1-b7f1-b2d623d56111 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:08.624879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf3bf8-dd75-4ee1-b7f1-b2d623d56111 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:08.632437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf3bf8-dd75-4ee1-b7f1-b2d623d56111 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:08.633877 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf3bf8-dd75-4ee1-b7f1-b2d623d56111 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:08.641139 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae562e9d-45c7-4ae7-879e-fe10a8b496c6 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:08.648789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-2bbf3bf8-dd75-4ee1-b7f1-b2d623d56111 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:08.648789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:08.648789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:08.648789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:08.648789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:08.648789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:08.648789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:08.648789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:08.648789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:08.648789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:08.648789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:08.648789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:08.648789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:08.648789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:08.648789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:08.649909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:08.649909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:08.649909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:08.649909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:08.649909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:08.649909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:08.649909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:08.649909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:08.649909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:08.649909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:08.649909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:08.649909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:08.649909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:08.649909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:08.649909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:08.649909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:08.649909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:08.650993 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:08.650993 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:08.650993 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:08.650993 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:08.650993 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:08.650993 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:08.650993 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:08.650993 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:08.650993 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 357/709] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:08.655907 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf3bf8-dd75-4ee1-b7f1-b2d623d56111 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:08.684908 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983521f-3d17-41db-9145-388b93bdb402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:08.686084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983521f-3d17-41db-9145-388b93bdb402 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:08.686507 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983521f-3d17-41db-9145-388b93bdb402 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:08.686920 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983521f-3d17-41db-9145-388b93bdb402 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:08.698850 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f983521f-3d17-41db-9145-388b93bdb402 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:08.700756 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f983521f-3d17-41db-9145-388b93bdb402 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:08.722527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-f983521f-3d17-41db-9145-388b93bdb402 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:08.722527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:08.722527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:08.722527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:08.722527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:08.722527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:08.722527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:08.722527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:08.722527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:08.722527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:08.722527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:08.722527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:08.722527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:08.722527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:08.722527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:08.723669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:08.723669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:08.723669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:08.723669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:08.723669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:08.723669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:08.723669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:08.723669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:08.723669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:08.723669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:08.723669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:08.723669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:08.723669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:08.723669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:08.723669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:08.723669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:08.723669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:08.724820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:08.724820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:08.724820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:08.724820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:08.728624 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 353/710] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:08.735027 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983521f-3d17-41db-9145-388b93bdb402 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:08.772935 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf3bf8-dd75-4ee1-b7f1-b2d623d56111 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:08.811370 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6a7a4-db5e-4a96-923c-a0855438f6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:08.814496 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6a7a4-db5e-4a96-923c-a0855438f6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:08.815015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6a7a4-db5e-4a96-923c-a0855438f6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:08.816265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6a7a4-db5e-4a96-923c-a0855438f6f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:08.817825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6a7a4-db5e-4a96-923c-a0855438f6f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:08.819191 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6a7a4-db5e-4a96-923c-a0855438f6f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:08.825499 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6a7a4-db5e-4a96-923c-a0855438f6f5 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:08.840930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 358/711] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:25:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:08.846303 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6a7a4-db5e-4a96-923c-a0855438f6f5 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:08.888866 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:08.931149 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983521f-3d17-41db-9145-388b93bdb402 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:08.952269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:08.986184 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bbc7f-2f9d-456f-bf91-7df2e3e9860c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:08.989366 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bbc7f-2f9d-456f-bf91-7df2e3e9860c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:08.989912 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bbc7f-2f9d-456f-bf91-7df2e3e9860c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:08.990379 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bbc7f-2f9d-456f-bf91-7df2e3e9860c None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:08.991322 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8bbc7f-2f9d-456f-bf91-7df2e3e9860c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:08.992590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8bbc7f-2f9d-456f-bf91-7df2e3e9860c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:08.994408 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8bbc7f-2f9d-456f-bf91-7df2e3e9860c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:09.021383 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 354/712] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:09.026974 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bbc7f-2f9d-456f-bf91-7df2e3e9860c None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:09.076375 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6a7a4-db5e-4a96-923c-a0855438f6f5 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:09.130227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265f25b-e377-414b-8329-e65d77d207a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:09.130889 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265f25b-e377-414b-8329-e65d77d207a7 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:09.131171 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265f25b-e377-414b-8329-e65d77d207a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:09.131446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265f25b-e377-414b-8329-e65d77d207a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:09.132310 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6265f25b-e377-414b-8329-e65d77d207a7 None admin] RBAC: Adding query filter params (role.id=95eccd30e39b4105ac6e8c0d9f973951, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=fede9e52ad274285b09d748a684ede38) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:09.132622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6265f25b-e377-414b-8329-e65d77d207a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:09.134048 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6265f25b-e377-414b-8329-e65d77d207a7 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:09.141196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 359/713] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:25:08 2022] GET /identity/v3/role_assignments?user.id=fede9e52ad274285b09d748a684ede38&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=95eccd30e39b4105ac6e8c0d9f973951 => generated 608 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:09.195637 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bbc7f-2f9d-456f-bf91-7df2e3e9860c None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:09.234212 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83680f-2b33-4f8c-af1b-07030a8c123f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:09.234891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83680f-2b33-4f8c-af1b-07030a8c123f None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:09.235175 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83680f-2b33-4f8c-af1b-07030a8c123f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:09.235453 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83680f-2b33-4f8c-af1b-07030a8c123f None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:09.236197 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83680f-2b33-4f8c-af1b-07030a8c123f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:09.238272 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83680f-2b33-4f8c-af1b-07030a8c123f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:09.265218 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 355/714] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jun 6 10:25:09 2022] POST /identity/v3/users => generated 254 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:25:09.479580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44196210-3c84-4d05-b3ff-b18e7801ce86 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:09.479952 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44196210-3c84-4d05-b3ff-b18e7801ce86 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:09.480164 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44196210-3c84-4d05-b3ff-b18e7801ce86 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:09.481918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 360/715] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:09.487806 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229ef4e-f789-438c-8807-10afc967b272 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:09.488227 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229ef4e-f789-438c-8807-10afc967b272 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:09.488398 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229ef4e-f789-438c-8807-10afc967b272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:09.510100 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-0229ef4e-f789-438c-8807-10afc967b272 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:09.511335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0229ef4e-f789-438c-8807-10afc967b272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:09.543799 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:09.598881 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 356/716] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:25:09.621665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44196210-3c84-4d05-b3ff-b18e7801ce86 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:09.730945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:09.751183 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44196210-3c84-4d05-b3ff-b18e7801ce86 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:09.787017 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca50f76-6011-4f22-8674-78090eb868cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:09.787890 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca50f76-6011-4f22-8674-78090eb868cc None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:09.788111 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca50f76-6011-4f22-8674-78090eb868cc None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:09.788387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca50f76-6011-4f22-8674-78090eb868cc None admin] PATH_INFO: `/v3/services/network` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:09.789045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca50f76-6011-4f22-8674-78090eb868cc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:09.790399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca50f76-6011-4f22-8674-78090eb868cc None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:09.804583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-cca50f76-6011-4f22-8674-78090eb868cc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 10:25:09.804583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:09.804583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:09.804583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:09.804583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:09.804583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:09.804583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:09.804583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:09.804583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 10:25:09.804583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:09.804583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:09.804583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:09.804583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:09.804583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:09.804583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:09.804583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:09.804583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:09.805622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:09.805622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:09.805622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:09.805622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:09.805622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:09.805622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:09.805622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:09.805622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:09.805622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:09.805622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:09.805622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:09.805622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:09.805622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:09.805622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:09.805622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:09.805622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:09.806875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:09.806875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:09.806875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:09.806875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:09.806875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:09.806875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:09.806875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:09.806875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:09.806875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:09.806875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:09.806875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:09.806875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:09.806875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 10:25:09.806875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:09.806875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 361/717] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:09 2022] GET /identity/v3/services/network => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:09.811808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229ef4e-f789-438c-8807-10afc967b272 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:09.959124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229ef4e-f789-438c-8807-10afc967b272 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:10.006272 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906de867-e630-4bab-91cc-58f6f52997a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:10.006874 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906de867-e630-4bab-91cc-58f6f52997a7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:10.007152 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906de867-e630-4bab-91cc-58f6f52997a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:10.007422 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906de867-e630-4bab-91cc-58f6f52997a7 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:10.008164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906de867-e630-4bab-91cc-58f6f52997a7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:10.008470 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906de867-e630-4bab-91cc-58f6f52997a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:10.010098 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906de867-e630-4bab-91cc-58f6f52997a7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:10.022691 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 357/718] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jun 6 10:25:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:10.028429 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca50f76-6011-4f22-8674-78090eb868cc None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:10.189996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca50f76-6011-4f22-8674-78090eb868cc None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:10.240915 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ace6d7-63b7-4f90-ade7-82618ff5ff02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:10.241958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ace6d7-63b7-4f90-ade7-82618ff5ff02 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:10.242467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ace6d7-63b7-4f90-ade7-82618ff5ff02 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:10.242934 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ace6d7-63b7-4f90-ade7-82618ff5ff02 None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:10.243863 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ace6d7-63b7-4f90-ade7-82618ff5ff02 None admin] RBAC: Adding query filter params (type=network) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:10.244365 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ace6d7-63b7-4f90-ade7-82618ff5ff02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:10.245916 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ace6d7-63b7-4f90-ade7-82618ff5ff02 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:10.257215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 362/719] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jun 6 10:25:10 2022] GET /identity/v3/services?type=network => generated 124 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:10.540901 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe9e08-73b7-4b41-a302-5872ee3ac878 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:10.541093 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe9e08-73b7-4b41-a302-5872ee3ac878 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:10.541381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe9e08-73b7-4b41-a302-5872ee3ac878 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:10.543051 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 358/720] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:10.548484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed015f6-18c3-4bd8-b5e2-acdcd2fbf036 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:10.548756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed015f6-18c3-4bd8-b5e2-acdcd2fbf036 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:10.549033 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed015f6-18c3-4bd8-b5e2-acdcd2fbf036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:10.572612 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-9ed015f6-18c3-4bd8-b5e2-acdcd2fbf036 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:10.573795 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-9ed015f6-18c3-4bd8-b5e2-acdcd2fbf036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:10.577382 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:10.655060 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 363/721] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:25:10.691983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe9e08-73b7-4b41-a302-5872ee3ac878 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:10.733152 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:10.831834 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe9e08-73b7-4b41-a302-5872ee3ac878 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:10.878913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284b923-d84a-41de-9a1b-06de79e6d494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:10.879725 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284b923-d84a-41de-9a1b-06de79e6d494 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:10.880028 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284b923-d84a-41de-9a1b-06de79e6d494 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:10.880299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284b923-d84a-41de-9a1b-06de79e6d494 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:10.896670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9284b923-d84a-41de-9a1b-06de79e6d494 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:25:10.904894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-9284b923-d84a-41de-9a1b-06de79e6d494 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:25:10.904894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:10.904894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:10.904894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:10.904894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:10.904894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:10.904894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:10.904894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:10.904894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:10.904894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:10.904894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:10.904894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:10.904894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:25:10.904894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:25:10.904894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:25:10.904894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:25:10.905822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:25:10.905822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:25:10.905822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:25:10.905822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:25:10.905822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:10.905822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:10.905822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:10.905822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:10.905822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:10.905822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:10.905822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:10.905822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:10.905822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:10.905822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:10.905822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:10.905822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:10.906657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:10.906657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:10.906657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:10.906657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:10.906657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:25:10.906657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:25:10.906657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:25:10.906657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:25:10.906657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:25:10.906657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:25:10.906657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:25:10.906657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:10.908676 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 359/722] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:25:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:10.917155 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed015f6-18c3-4bd8-b5e2-acdcd2fbf036 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:11.054689 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed015f6-18c3-4bd8-b5e2-acdcd2fbf036 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:11.089043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03449dda-8d3e-4766-a6ae-7c79d414f808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:11.089682 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03449dda-8d3e-4766-a6ae-7c79d414f808 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:11.089969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03449dda-8d3e-4766-a6ae-7c79d414f808 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:11.090245 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03449dda-8d3e-4766-a6ae-7c79d414f808 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:11.090990 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03449dda-8d3e-4766-a6ae-7c79d414f808 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:11.091295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03449dda-8d3e-4766-a6ae-7c79d414f808 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:11.092678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03449dda-8d3e-4766-a6ae-7c79d414f808 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:11.102628 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 364/723] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:25:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:11.108847 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284b923-d84a-41de-9a1b-06de79e6d494 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:11.169669 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ce935-78ff-4a63-bda4-03f8a2345f29 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:11.169958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ce935-78ff-4a63-bda4-03f8a2345f29 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:11.171064 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ce935-78ff-4a63-bda4-03f8a2345f29 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:11.171862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 365/724] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:11.177116 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0489b-5702-4c26-a91b-293fb11c173d None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:11.177643 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0489b-5702-4c26-a91b-293fb11c173d None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:11.178085 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0489b-5702-4c26-a91b-293fb11c173d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:11.197263 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:11.209026 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-e5e0489b-5702-4c26-a91b-293fb11c173d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:11.212613 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-e5e0489b-5702-4c26-a91b-293fb11c173d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:11.278985 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:11.307759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284b923-d84a-41de-9a1b-06de79e6d494 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:11.341147 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 366/725] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:25:11.361158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7de05-e2ec-42b9-93b4-961305c95a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:11.362350 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7de05-e2ec-42b9-93b4-961305c95a2c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:11.362839 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7de05-e2ec-42b9-93b4-961305c95a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:11.363316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7de05-e2ec-42b9-93b4-961305c95a2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:11.367828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0489b-5702-4c26-a91b-293fb11c173d None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:11.385883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7de05-e2ec-42b9-93b4-961305c95a2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:11.387340 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7de05-e2ec-42b9-93b4-961305c95a2c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:11.416541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-b5d7de05-e2ec-42b9-93b4-961305c95a2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:11.416541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:11.416541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:11.416541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:11.416541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:11.416541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:11.416541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:11.416541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:11.416541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:11.416541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:11.416541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:11.416541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:11.416541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:11.416541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:11.416541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:11.417487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:11.417487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:11.417487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:11.417487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:11.417487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:11.417487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:11.417487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:11.417487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:11.417487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:11.417487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:11.417487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:11.417487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:11.417487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:11.417487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:11.417487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:11.417487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:11.417487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:11.418389 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:11.418389 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:11.418389 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:11.418389 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:11.422349 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 360/726] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:11.423611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7de05-e2ec-42b9-93b4-961305c95a2c None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:11.530552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7de05-e2ec-42b9-93b4-961305c95a2c None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:11.547572 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0489b-5702-4c26-a91b-293fb11c173d None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:11.567277 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f938802-c1f1-4122-bfed-7c6d9e10ffca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:11.568311 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f938802-c1f1-4122-bfed-7c6d9e10ffca None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:11.568798 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f938802-c1f1-4122-bfed-7c6d9e10ffca None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:11.569270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f938802-c1f1-4122-bfed-7c6d9e10ffca None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:11.570192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f938802-c1f1-4122-bfed-7c6d9e10ffca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:11.570731 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f938802-c1f1-4122-bfed-7c6d9e10ffca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:11.572336 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f938802-c1f1-4122-bfed-7c6d9e10ffca None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:11.579582 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:11.586712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 361/727] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:11.591965 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f938802-c1f1-4122-bfed-7c6d9e10ffca None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:11.612841 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:11.630500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1b0dd-9898-436a-a166-3a10a7dbdaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:11.631169 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1b0dd-9898-436a-a166-3a10a7dbdaeb None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:11.631449 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1b0dd-9898-436a-a166-3a10a7dbdaeb None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:11.631728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1b0dd-9898-436a-a166-3a10a7dbdaeb None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:11.632467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1b0dd-9898-436a-a166-3a10a7dbdaeb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:11.634472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1b0dd-9898-436a-a166-3a10a7dbdaeb None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:11.681440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 367/728] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jun 6 10:25:11 2022] POST /identity/v3/services => generated 251 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:25:11.712198 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f938802-c1f1-4122-bfed-7c6d9e10ffca None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:11.757977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7f127-f517-4610-91bf-e4679b16b823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:11.759132 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7f127-f517-4610-91bf-e4679b16b823 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:11.759648 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7f127-f517-4610-91bf-e4679b16b823 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:11.760215 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7f127-f517-4610-91bf-e4679b16b823 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:11.768886 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de7f127-f517-4610-91bf-e4679b16b823 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:11.770757 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de7f127-f517-4610-91bf-e4679b16b823 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:11.787979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-6de7f127-f517-4610-91bf-e4679b16b823 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 10:25:11.787979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:11.787979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:11.787979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:11.787979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:11.787979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:11.787979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:11.787979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:11.787979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:11.787979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:11.787979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:11.787979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:11.787979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:25:11.787979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:25:11.787979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:25:11.787979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:25:11.788989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:11.788989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:11.788989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:11.788989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:11.788989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:11.788989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:11.788989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:11.788989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:11.788989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:11.788989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:11.788989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:11.788989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:11.788989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:11.788989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:11.788989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:11.788989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:11.788989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:11.788989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:11.789917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:11.789917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:11.789917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:25:11.789917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:25:11.789917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:25:11.789917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:25:11.789917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:25:11.789917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:25:11.789917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 10:25:11.789917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:11.792672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 362/729] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jun 6 10:25:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:11.800032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1b0dd-9898-436a-a166-3a10a7dbdaeb None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:11.921038 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1b0dd-9898-436a-a166-3a10a7dbdaeb None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:11.963589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4181d2f-01c0-473c-bce7-e66a6caa39e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:11.964306 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4181d2f-01c0-473c-bce7-e66a6caa39e1 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:11.964587 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4181d2f-01c0-473c-bce7-e66a6caa39e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:11.964858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4181d2f-01c0-473c-bce7-e66a6caa39e1 None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:11.965715 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4181d2f-01c0-473c-bce7-e66a6caa39e1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:11.966023 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4181d2f-01c0-473c-bce7-e66a6caa39e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:11.967472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4181d2f-01c0-473c-bce7-e66a6caa39e1 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:11.992746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 368/730] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jun 6 10:25:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:11.998017 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7f127-f517-4610-91bf-e4679b16b823 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:12.071913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:12.121092 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7f127-f517-4610-91bf-e4679b16b823 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:12.162345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9824b22-932e-4b2f-9ce5-c473c968438d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:12.163628 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9824b22-932e-4b2f-9ce5-c473c968438d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:12.164150 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9824b22-932e-4b2f-9ce5-c473c968438d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:12.164641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9824b22-932e-4b2f-9ce5-c473c968438d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:12.178400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9824b22-932e-4b2f-9ce5-c473c968438d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:12.180275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9824b22-932e-4b2f-9ce5-c473c968438d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:12.201282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-d9824b22-932e-4b2f-9ce5-c473c968438d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:12.201282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:12.201282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:12.201282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:12.201282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:12.201282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:12.201282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:12.201282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:12.201282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:12.201282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:12.201282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:12.201282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:12.201282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:12.201282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:12.201282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:12.202276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:12.202276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:12.202276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:12.202276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:12.202276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:12.202276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:12.202276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:12.202276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:12.202276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:12.202276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:12.202276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:12.202276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:12.202276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:12.202276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:12.202276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:12.202276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:12.202276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:12.203280 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:12.203280 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:12.203280 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:12.203280 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:12.206354 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 363/731] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:12.217720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4181d2f-01c0-473c-bce7-e66a6caa39e1 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:12.282510 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:12.352334 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4181d2f-01c0-473c-bce7-e66a6caa39e1 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:12.396065 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761eb87-c4ba-4a33-b0e5-783f2e0e8e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:12.396750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761eb87-c4ba-4a33-b0e5-783f2e0e8e4f None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:12.397033 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761eb87-c4ba-4a33-b0e5-783f2e0e8e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:12.397308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761eb87-c4ba-4a33-b0e5-783f2e0e8e4f None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:12.398039 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c761eb87-c4ba-4a33-b0e5-783f2e0e8e4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:12.398596 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c761eb87-c4ba-4a33-b0e5-783f2e0e8e4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:12.399994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c761eb87-c4ba-4a33-b0e5-783f2e0e8e4f None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:12.419962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 369/732] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:12.426025 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9824b22-932e-4b2f-9ce5-c473c968438d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:12.512893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa522d-7456-4733-8921-6697b1852983 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:12.513228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa522d-7456-4733-8921-6697b1852983 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:12.513451 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa522d-7456-4733-8921-6697b1852983 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:12.516172 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 370/733] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:12.520467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f70ea-c5a0-4770-a85c-aa5600e6aa33 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:12.520750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f70ea-c5a0-4770-a85c-aa5600e6aa33 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:12.521028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f70ea-c5a0-4770-a85c-aa5600e6aa33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:12.538640 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-f26f70ea-c5a0-4770-a85c-aa5600e6aa33 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:12.539846 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-f26f70ea-c5a0-4770-a85c-aa5600e6aa33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:12.576554 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9824b22-932e-4b2f-9ce5-c473c968438d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:12.585232 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:12.631944 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663860f-a7c7-4057-becf-5f0fd28282a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:12.632748 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663860f-a7c7-4057-becf-5f0fd28282a9 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:12.633026 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663860f-a7c7-4057-becf-5f0fd28282a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:12.633297 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663860f-a7c7-4057-becf-5f0fd28282a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:12.645505 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663860f-a7c7-4057-becf-5f0fd28282a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:12.646952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663860f-a7c7-4057-becf-5f0fd28282a9 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:12.669655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-6663860f-a7c7-4057-becf-5f0fd28282a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:12.669655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:12.669655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:12.669655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:12.669655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:12.669655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:12.669655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:12.669655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:12.669655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:12.669655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:12.669655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:12.669655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:12.669655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:12.669655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:12.669655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:12.670618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:12.670618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:12.670618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:12.670618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:12.670618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:12.670618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:12.670618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:12.670618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:12.670618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:12.670618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:12.670618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:12.670618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:12.670618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:12.670618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:12.670618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:12.670618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:12.670618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:12.671514 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:12.671514 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:12.671514 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:12.671514 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:12.671514 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:12.671514 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:12.671514 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:12.671514 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:12.671514 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 364/734] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:12.680789 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663860f-a7c7-4057-becf-5f0fd28282a9 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:12.688013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 371/735] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:12 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 10:25:12.707748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160de93e-3ecf-40c5-a2b9-f94356089e77 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:12.708371 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160de93e-3ecf-40c5-a2b9-f94356089e77 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:12.708877 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160de93e-3ecf-40c5-a2b9-f94356089e77 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:12.710734 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 372/736] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:25:12.715640 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1fc89-36e2-443a-b450-f5961ab06afe None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:12.716183 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1fc89-36e2-443a-b450-f5961ab06afe None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:12.716656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1fc89-36e2-443a-b450-f5961ab06afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:12.736209 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-5be1fc89-36e2-443a-b450-f5961ab06afe None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:12.738001 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-5be1fc89-36e2-443a-b450-f5961ab06afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:12.775169 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:12.811036 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 373/737] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:12 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:25:12.811919 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1fc89-36e2-443a-b450-f5961ab06afe None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:12.818119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663860f-a7c7-4057-becf-5f0fd28282a9 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:12.873021 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74693f-c5f3-43c8-88f1-58756c3fbeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:12.873969 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74693f-c5f3-43c8-88f1-58756c3fbeca None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:12.874665 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74693f-c5f3-43c8-88f1-58756c3fbeca None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:12.874948 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74693f-c5f3-43c8-88f1-58756c3fbeca None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:12.876053 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74693f-c5f3-43c8-88f1-58756c3fbeca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:12.876395 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74693f-c5f3-43c8-88f1-58756c3fbeca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:12.878789 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74693f-c5f3-43c8-88f1-58756c3fbeca None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:12.895958 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 365/738] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:25:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:12.897177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74693f-c5f3-43c8-88f1-58756c3fbeca None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:12.920054 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1fc89-36e2-443a-b450-f5961ab06afe None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:12.936685 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:12.954902 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475091ec-cff5-473f-8eac-fc4abdab97f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:12.956233 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475091ec-cff5-473f-8eac-fc4abdab97f2 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:12.956535 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475091ec-cff5-473f-8eac-fc4abdab97f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:12.956827 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475091ec-cff5-473f-8eac-fc4abdab97f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:12.966986 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475091ec-cff5-473f-8eac-fc4abdab97f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:25:12.975409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-475091ec-cff5-473f-8eac-fc4abdab97f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:25:12.975409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:12.975409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:12.975409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:12.975409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:12.975409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:12.975409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:12.975409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:12.975409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:12.975409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:12.975409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:12.975409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:12.975409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:25:12.975409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:25:12.975409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:25:12.975409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:25:12.976393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:25:12.976393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:25:12.976393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:25:12.976393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:25:12.976393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:12.976393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:12.976393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:12.976393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:12.976393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:12.976393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:12.976393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:12.976393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:12.976393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:12.976393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:12.976393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:12.976393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:12.977254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:12.977254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:12.977254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:12.977254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:12.977254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:25:12.977254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:25:12.977254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:25:12.977254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:25:12.977254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:25:12.977254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:25:12.977254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:25:12.977254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:12.977254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 374/739] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:25:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:12.977934 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475091ec-cff5-473f-8eac-fc4abdab97f2 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:13.013616 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:13.017541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74693f-c5f3-43c8-88f1-58756c3fbeca None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:13.054584 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344c116-e566-4163-be00-bd8013a0937f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:13.055319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344c116-e566-4163-be00-bd8013a0937f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:13.055578 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344c116-e566-4163-be00-bd8013a0937f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:13.055877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344c116-e566-4163-be00-bd8013a0937f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:13.067850 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344c116-e566-4163-be00-bd8013a0937f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:25:13.075745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-a344c116-e566-4163-be00-bd8013a0937f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:25:13.075745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:13.075745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:13.075745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:13.075745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:13.075745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:13.075745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:13.075745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:13.075745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:13.075745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:13.075745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:13.075745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:13.075745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:25:13.075745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:25:13.075745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:25:13.075745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:25:13.076683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:25:13.076683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:25:13.076683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:25:13.076683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:25:13.076683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:13.076683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:13.076683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:13.076683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:13.076683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:13.076683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:13.076683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:13.076683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:13.076683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:13.076683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:13.076683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:13.076683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:13.077526 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:13.077526 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:13.077526 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:13.077526 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:13.077526 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:25:13.077526 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:25:13.077526 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:25:13.077526 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:25:13.077526 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:25:13.077526 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:25:13.077526 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:25:13.077526 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:13.077526 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 366/740] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:25:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:13.078729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316935f-0706-40a2-aa78-e662cb03bc6c None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:13.078829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316935f-0706-40a2-aa78-e662cb03bc6c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:13.079031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316935f-0706-40a2-aa78-e662cb03bc6c None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:13.084065 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 367/741] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:13.085220 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316935f-0706-40a2-aa78-e662cb03bc6c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:13.095418 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475091ec-cff5-473f-8eac-fc4abdab97f2 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:13.131309 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902940ea-8811-4057-8085-e70f8f1b8d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:13.132223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902940ea-8811-4057-8085-e70f8f1b8d7d None admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:13.132512 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902940ea-8811-4057-8085-e70f8f1b8d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:13.132791 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902940ea-8811-4057-8085-e70f8f1b8d7d None admin] PATH_INFO: `/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/ed8a40e4f1034a8e99d0eb505e448db9/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:13.135174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902940ea-8811-4057-8085-e70f8f1b8d7d None admin] RBAC: Authorizing `identity:create_grant(project_id=1f082d7ed3694fd9998623f3d4a5fea9, user_id=ed8a40e4f1034a8e99d0eb505e448db9, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:13.136994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902940ea-8811-4057-8085-e70f8f1b8d7d None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:13.159823 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 375/742] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:25:12 2022] PUT /identity/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/ed8a40e4f1034a8e99d0eb505e448db9/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:25:13.159823 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902940ea-8811-4057-8085-e70f8f1b8d7d None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:13.206367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:13.230005 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316935f-0706-40a2-aa78-e662cb03bc6c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:13.280585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:13.287972 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b5909-b8cf-4da7-bac8-1f59890f56ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:13.288675 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b5909-b8cf-4da7-bac8-1f59890f56ba None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:13.288961 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b5909-b8cf-4da7-bac8-1f59890f56ba None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:13.289411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b5909-b8cf-4da7-bac8-1f59890f56ba None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:13.290534 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b5909-b8cf-4da7-bac8-1f59890f56ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:13.290992 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b5909-b8cf-4da7-bac8-1f59890f56ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:13.292809 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b5909-b8cf-4da7-bac8-1f59890f56ba None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:13.293395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902940ea-8811-4057-8085-e70f8f1b8d7d None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:13.308368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 368/743] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:25:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:13.309950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b8be9-6e25-470c-9c98-b91fd6b4384a None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:13.310758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b8be9-6e25-470c-9c98-b91fd6b4384a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:13.311044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b8be9-6e25-470c-9c98-b91fd6b4384a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:13.337242 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-0e9b8be9-6e25-470c-9c98-b91fd6b4384a None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:13.339710 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0e9b8be9-6e25-470c-9c98-b91fd6b4384a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:13.343633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c89c9e-6367-4c32-b34b-36b4cb41a50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:13.344941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c89c9e-6367-4c32-b34b-36b4cb41a50b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:13.344941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c89c9e-6367-4c32-b34b-36b4cb41a50b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:13.345142 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c89c9e-6367-4c32-b34b-36b4cb41a50b None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:13.345945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c89c9e-6367-4c32-b34b-36b4cb41a50b None admin] RBAC: Adding query filter params (name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:13.346985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c89c9e-6367-4c32-b34b-36b4cb41a50b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:13.348501 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c89c9e-6367-4c32-b34b-36b4cb41a50b None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:13.363824 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 376/744] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jun 6 10:25:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:13.363824 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c89c9e-6367-4c32-b34b-36b4cb41a50b None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:13.432257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 369/745] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:13 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 10:25:13.432919 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b8be9-6e25-470c-9c98-b91fd6b4384a None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:13.501368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c89c9e-6367-4c32-b34b-36b4cb41a50b None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:13.523554 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:13.539997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e39cc-ad94-4182-9550-49779f2bed52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:13.541188 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e39cc-ad94-4182-9550-49779f2bed52 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:13.541697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e39cc-ad94-4182-9550-49779f2bed52 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:13.542118 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e39cc-ad94-4182-9550-49779f2bed52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:13.547391 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b8be9-6e25-470c-9c98-b91fd6b4384a None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:13.554773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e39cc-ad94-4182-9550-49779f2bed52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:13.556523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e39cc-ad94-4182-9550-49779f2bed52 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:13.576968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-430e39cc-ad94-4182-9550-49779f2bed52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:13.576968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:13.576968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:13.576968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:13.576968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:13.576968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:13.576968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:13.576968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:13.576968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:13.576968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:13.576968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:13.576968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:13.576968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:13.576968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:13.576968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:13.578059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:13.578059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:13.578059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:13.578059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:13.578059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:13.578059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:13.578059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:13.578059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:13.578059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:13.578059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:13.578059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:13.578059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:13.578059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:13.578059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:13.578059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:13.578059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:13.578059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:13.579165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:13.579165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:13.579165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:13.579165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:13.581601 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 377/746] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:13.582599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e39cc-ad94-4182-9550-49779f2bed52 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:13.588548 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9a7af-05b8-4c0a-97a9-b247a8157b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:13.590547 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9a7af-05b8-4c0a-97a9-b247a8157b65 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:13.591019 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9a7af-05b8-4c0a-97a9-b247a8157b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:13.591450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9a7af-05b8-4c0a-97a9-b247a8157b65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:13.604145 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:13.605258 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9a7af-05b8-4c0a-97a9-b247a8157b65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:13.606908 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9a7af-05b8-4c0a-97a9-b247a8157b65 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:13.626072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-c9f9a7af-05b8-4c0a-97a9-b247a8157b65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:13.626072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:13.626072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:13.626072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:13.626072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:13.626072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:13.626072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:13.626072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:13.626072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:13.626072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:13.626072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:13.626072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:13.626072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:13.626072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:13.626072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:13.628238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:13.628238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:13.628238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:13.628238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:13.628238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:13.628238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:13.628238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:13.628238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:13.628238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:13.628238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:13.628238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:13.628238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:13.628238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:13.628238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:13.628238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:13.628238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:13.628238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:13.629243 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:13.629243 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:13.629243 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:13.629243 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:13.634308 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 370/747] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:13.635318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9a7af-05b8-4c0a-97a9-b247a8157b65 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:13.723381 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e39cc-ad94-4182-9550-49779f2bed52 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:13.769388 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f8f6a-6fd2-4e44-a040-061be4113764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:13.770634 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f8f6a-6fd2-4e44-a040-061be4113764 None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:13.771095 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f8f6a-6fd2-4e44-a040-061be4113764 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:13.771733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f8f6a-6fd2-4e44-a040-061be4113764 None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:13.773701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f8f6a-6fd2-4e44-a040-061be4113764 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:13.775291 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f8f6a-6fd2-4e44-a040-061be4113764 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:13.775541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9a7af-05b8-4c0a-97a9-b247a8157b65 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:13.800009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 378/748] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jun 6 10:25:13 2022] POST /identity/v3/services => generated 240 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:25:13.800009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f8f6a-6fd2-4e44-a040-061be4113764 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:13.832158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:13.854616 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a86ba-e47e-4564-8f9c-6149a9e5ca38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:13.855288 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a86ba-e47e-4564-8f9c-6149a9e5ca38 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:13.855572 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a86ba-e47e-4564-8f9c-6149a9e5ca38 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:13.855883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a86ba-e47e-4564-8f9c-6149a9e5ca38 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:13.856618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a86ba-e47e-4564-8f9c-6149a9e5ca38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:13.856986 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a86ba-e47e-4564-8f9c-6149a9e5ca38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:13.858651 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a86ba-e47e-4564-8f9c-6149a9e5ca38 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:13.885837 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 371/749] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:13.893706 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a86ba-e47e-4564-8f9c-6149a9e5ca38 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:13.939115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:14.018018 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f8f6a-6fd2-4e44-a040-061be4113764 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:14.094447 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a86ba-e47e-4564-8f9c-6149a9e5ca38 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:14.099189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad205ba-8aba-459a-815e-4041a0b6368a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:14.100229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad205ba-8aba-459a-815e-4041a0b6368a None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:14.100711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad205ba-8aba-459a-815e-4041a0b6368a None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:14.101209 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad205ba-8aba-459a-815e-4041a0b6368a None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:14.102243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad205ba-8aba-459a-815e-4041a0b6368a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:14.102742 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad205ba-8aba-459a-815e-4041a0b6368a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:14.104433 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad205ba-8aba-459a-815e-4041a0b6368a None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:14.120527 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 379/750] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:14.128129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad205ba-8aba-459a-815e-4041a0b6368a None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:14.136570 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209abd3-50fa-470f-b7bc-78e489855eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:14.137375 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209abd3-50fa-470f-b7bc-78e489855eba None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:14.137655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209abd3-50fa-470f-b7bc-78e489855eba None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:14.137931 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209abd3-50fa-470f-b7bc-78e489855eba None admin] PATH_INFO: `/v3/users/swift` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:14.147554 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209abd3-50fa-470f-b7bc-78e489855eba None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:14.149291 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209abd3-50fa-470f-b7bc-78e489855eba None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:14.165903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-c209abd3-50fa-470f-b7bc-78e489855eba None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 10:25:14.165903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:14.165903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:14.165903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:14.165903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:14.165903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:14.165903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:14.165903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:14.165903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:14.165903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:14.165903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:14.165903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:14.165903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:25:14.165903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:25:14.165903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:25:14.165903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:25:14.166904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:14.166904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:14.166904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:14.166904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:14.166904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:14.166904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:14.166904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:14.166904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:14.166904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:14.166904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:14.166904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:14.166904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:14.166904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:14.166904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:14.166904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:14.166904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:14.166904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:14.166904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:14.168450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:14.168450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:14.168450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:25:14.168450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:25:14.168450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:25:14.168450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:25:14.168450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:25:14.168450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:25:14.168450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 10:25:14.168450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:14.168450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 372/751] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:25:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:14.173528 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209abd3-50fa-470f-b7bc-78e489855eba None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:14.231499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:14.275951 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad205ba-8aba-459a-815e-4041a0b6368a None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:14.296387 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209abd3-50fa-470f-b7bc-78e489855eba None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:14.309390 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:14.327429 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa4c88-433d-4eb0-ae98-45b84035c86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:14.328732 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa4c88-433d-4eb0-ae98-45b84035c86e None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:14.329475 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa4c88-433d-4eb0-ae98-45b84035c86e None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:14.330391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa4c88-433d-4eb0-ae98-45b84035c86e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:14.339155 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75018a5-963f-4b67-b373-70aff51217b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:14.339878 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75018a5-963f-4b67-b373-70aff51217b1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:14.340264 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75018a5-963f-4b67-b373-70aff51217b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:14.340435 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75018a5-963f-4b67-b373-70aff51217b1 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:14.341270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75018a5-963f-4b67-b373-70aff51217b1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:14.341577 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75018a5-963f-4b67-b373-70aff51217b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:14.342047 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa4c88-433d-4eb0-ae98-45b84035c86e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:14.343017 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75018a5-963f-4b67-b373-70aff51217b1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:14.343908 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa4c88-433d-4eb0-ae98-45b84035c86e None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:14.363565 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 373/752] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Jun 6 10:25:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:14.365472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-46aa4c88-433d-4eb0-ae98-45b84035c86e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 10:25:14.365472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:14.365472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:14.365472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:14.365472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:14.365472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:14.365472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:14.365472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:14.365472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:14.365472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:14.365472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:14.365472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:14.365472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:25:14.365472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:25:14.365472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:25:14.365472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:25:14.366384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:14.366384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:14.366384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:14.366384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:14.366384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:14.366384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:14.366384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:14.366384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:14.366384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:14.366384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:14.366384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:14.366384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:14.366384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:14.366384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:14.366384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:14.366384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:14.366384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:14.366384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:14.367322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:14.367322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:14.367322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:25:14.367322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:25:14.367322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:25:14.367322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:25:14.367322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:25:14.367322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:25:14.367322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 10:25:14.367322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:14.369994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 380/753] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jun 6 10:25:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:14.375744 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75018a5-963f-4b67-b373-70aff51217b1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:14.376362 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa4c88-433d-4eb0-ae98-45b84035c86e None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:14.501937 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa4c88-433d-4eb0-ae98-45b84035c86e None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:14.546588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a3186-b2be-42ea-9538-d557538e7b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:14.547241 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a3186-b2be-42ea-9538-d557538e7b0f None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:14.547519 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a3186-b2be-42ea-9538-d557538e7b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:14.547877 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a3186-b2be-42ea-9538-d557538e7b0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:14.609250 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75018a5-963f-4b67-b373-70aff51217b1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:14.611693 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:14.614966 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:14.635092 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a3186-b2be-42ea-9538-d557538e7b0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:14.637106 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a3186-b2be-42ea-9538-d557538e7b0f None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:14.662261 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe1ab2-998e-4f20-95ef-b3c9e3ce3bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:14.663230 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe1ab2-998e-4f20-95ef-b3c9e3ce3bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:14.664248 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe1ab2-998e-4f20-95ef-b3c9e3ce3bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:14.664687 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe1ab2-998e-4f20-95ef-b3c9e3ce3bd8 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:14.665945 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe1ab2-998e-4f20-95ef-b3c9e3ce3bd8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:14.666643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe1ab2-998e-4f20-95ef-b3c9e3ce3bd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:14.668981 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe1ab2-998e-4f20-95ef-b3c9e3ce3bd8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:14.694734 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 374/754] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Jun 6 10:25:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:14.696369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8ee72-7f7e-4601-afea-f2c83005204a None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:14.697295 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8ee72-7f7e-4601-afea-f2c83005204a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:14.697724 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8ee72-7f7e-4601-afea-f2c83005204a None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:14.699775 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 375/755] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:14.700686 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8ee72-7f7e-4601-afea-f2c83005204a None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:14.753485 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 381/756] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:25:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 10:25:14.754961 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f536786-e1a2-4eca-9a67-898db21268e6 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:14.755253 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f536786-e1a2-4eca-9a67-898db21268e6 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:14.755530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f536786-e1a2-4eca-9a67-898db21268e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:14.772576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-9f536786-e1a2-4eca-9a67-898db21268e6 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:14.774012 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-9f536786-e1a2-4eca-9a67-898db21268e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:14.830144 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8ee72-7f7e-4601-afea-f2c83005204a None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:14.866281 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5fcae7-ebdb-4744-b2ec-1df67ad79caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:14.866889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fcae7-ebdb-4744-b2ec-1df67ad79caf None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:14.867130 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fcae7-ebdb-4744-b2ec-1df67ad79caf None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:14.867409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fcae7-ebdb-4744-b2ec-1df67ad79caf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:14.871674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 382/757] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:14.872800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f536786-e1a2-4eca-9a67-898db21268e6 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:14.890021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:14.920669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:14.976695 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fcae7-ebdb-4744-b2ec-1df67ad79caf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:14.979045 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fcae7-ebdb-4744-b2ec-1df67ad79caf None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:15.003573 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f536786-e1a2-4eca-9a67-898db21268e6 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:15.043750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989381f-f0c4-4be5-aa6e-e69fa0a4ee8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:15.044571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989381f-f0c4-4be5-aa6e-e69fa0a4ee8d None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:15.044853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989381f-f0c4-4be5-aa6e-e69fa0a4ee8d None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:15.045165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989381f-f0c4-4be5-aa6e-e69fa0a4ee8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:15.055806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989381f-f0c4-4be5-aa6e-e69fa0a4ee8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:15.057258 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989381f-f0c4-4be5-aa6e-e69fa0a4ee8d None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:15.075862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-5989381f-f0c4-4be5-aa6e-e69fa0a4ee8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:15.075862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:15.075862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:15.075862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:15.075862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:15.075862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:15.075862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:15.075862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:15.075862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:15.075862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:15.075862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:15.075862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:15.075862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:15.075862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:15.075862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:15.076797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:15.076797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:15.076797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:15.076797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:15.076797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:15.076797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:15.076797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:15.076797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:15.076797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:15.076797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:15.076797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:15.076797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:15.076797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:15.076797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:15.076797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:15.076797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:15.076797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:15.077930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:15.077930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:15.077930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:15.077930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:15.077930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 383/758] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:15.078291 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989381f-f0c4-4be5-aa6e-e69fa0a4ee8d None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:15.126709 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 376/759] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:25:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 10:25:15.128688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5fcae7-ebdb-4744-b2ec-1df67ad79caf None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:15.155088 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:15.204790 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989381f-f0c4-4be5-aa6e-e69fa0a4ee8d None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:15.206227 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:15.248332 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5fcae7-ebdb-4744-b2ec-1df67ad79caf None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:15.255289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f24ac09-3db1-4c1c-b6ef-c3a48d2ac4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:15.256399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24ac09-3db1-4c1c-b6ef-c3a48d2ac4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:15.257392 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24ac09-3db1-4c1c-b6ef-c3a48d2ac4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:15.257845 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24ac09-3db1-4c1c-b6ef-c3a48d2ac4d0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:15.259109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f24ac09-3db1-4c1c-b6ef-c3a48d2ac4d0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:15.261114 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f24ac09-3db1-4c1c-b6ef-c3a48d2ac4d0 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:15.280528 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-8f24ac09-3db1-4c1c-b6ef-c3a48d2ac4d0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 10:25:15.280528 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:15.280528 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:15.280528 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:15.280528 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:15.280528 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:15.280528 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:15.280528 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:15.280528 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 10:25:15.280528 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:15.280528 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:15.280528 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:15.280528 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:15.280528 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:15.280528 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:15.280528 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:15.281525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:15.281525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:15.281525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:15.281525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:15.281525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:15.281525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:15.281525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:15.281525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:15.281525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:15.281525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:15.281525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:15.281525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:15.281525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:15.281525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:15.281525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:15.281525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:15.282364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:15.282364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:15.282364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:15.282364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:15.282364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:15.282364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:15.282364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:15.282364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:15.282364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:15.282364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:15.282364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:15.282364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:15.282364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:15.282364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 10:25:15.282364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:15.286423 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 384/760] 10.222.0.42 () {56 vars in 1129 bytes} [Mon Jun 6 10:25:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:15.287469 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f24ac09-3db1-4c1c-b6ef-c3a48d2ac4d0 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:15.295416 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01301a8-2087-4eb3-bb47-d127f2bef66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:15.296128 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01301a8-2087-4eb3-bb47-d127f2bef66b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:15.296417 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01301a8-2087-4eb3-bb47-d127f2bef66b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:15.296697 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01301a8-2087-4eb3-bb47-d127f2bef66b None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:15.297435 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01301a8-2087-4eb3-bb47-d127f2bef66b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:15.297744 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01301a8-2087-4eb3-bb47-d127f2bef66b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:15.299129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01301a8-2087-4eb3-bb47-d127f2bef66b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:15.314725 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 377/761] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:15.316617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01301a8-2087-4eb3-bb47-d127f2bef66b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:15.422563 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f24ac09-3db1-4c1c-b6ef-c3a48d2ac4d0 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:15.453088 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:15.460965 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88643c-2ced-4e6b-8a0f-37ef1e07f74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:15.461763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88643c-2ced-4e6b-8a0f-37ef1e07f74b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:15.462047 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88643c-2ced-4e6b-8a0f-37ef1e07f74b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:15.462324 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88643c-2ced-4e6b-8a0f-37ef1e07f74b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:15.463242 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01301a8-2087-4eb3-bb47-d127f2bef66b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:15.473859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88643c-2ced-4e6b-8a0f-37ef1e07f74b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:15.475339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88643c-2ced-4e6b-8a0f-37ef1e07f74b None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:15.491458 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:15.493777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-0c88643c-2ced-4e6b-8a0f-37ef1e07f74b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:15.493777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:15.493777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:15.493777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:15.493777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:15.493777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:15.493777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:15.493777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:15.493777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:15.493777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:15.493777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:15.493777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:15.493777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:15.493777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:15.493777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:15.496491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:15.496491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:15.496491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:15.496491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:15.496491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:15.496491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:15.496491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:15.496491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:15.496491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:15.496491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:15.496491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:15.496491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:15.496491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:15.496491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:15.496491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:15.496491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:15.496491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:15.497399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:15.497399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:15.497399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:15.497399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:15.497399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 385/762] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:15.497399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88643c-2ced-4e6b-8a0f-37ef1e07f74b None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:15.509765 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6fad7-143f-462b-86ba-e9de8d0758fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:15.511158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6fad7-143f-462b-86ba-e9de8d0758fe None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:15.511932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6fad7-143f-462b-86ba-e9de8d0758fe None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:15.512407 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6fad7-143f-462b-86ba-e9de8d0758fe None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:15.513597 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6fad7-143f-462b-86ba-e9de8d0758fe None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:15.514546 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6fad7-143f-462b-86ba-e9de8d0758fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:15.516648 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6fad7-143f-462b-86ba-e9de8d0758fe None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:15.524419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 378/763] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Jun 6 10:25:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:15.525509 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6fad7-143f-462b-86ba-e9de8d0758fe None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:15.613405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88643c-2ced-4e6b-8a0f-37ef1e07f74b None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:15.649697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb5ad2-24dd-4967-b56c-1310533345f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:15.650413 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb5ad2-24dd-4967-b56c-1310533345f4 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:15.650745 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb5ad2-24dd-4967-b56c-1310533345f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:15.651026 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb5ad2-24dd-4967-b56c-1310533345f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:15.657937 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb5ad2-24dd-4967-b56c-1310533345f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:15.660084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6fad7-143f-462b-86ba-e9de8d0758fe None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:15.660257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb5ad2-24dd-4967-b56c-1310533345f4 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:15.675977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-04eb5ad2-24dd-4967-b56c-1310533345f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:15.675977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:15.675977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:15.675977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:15.675977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:15.675977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:15.675977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:15.675977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:15.675977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:15.675977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:15.675977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:15.675977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:15.675977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:15.675977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:15.675977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:15.676917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:15.676917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:15.676917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:15.676917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:15.676917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:15.676917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:15.676917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:15.676917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:15.676917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:15.676917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:15.676917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:15.676917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:15.676917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:15.676917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:15.676917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:15.676917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:15.676917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:15.678437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:15.678437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:15.678437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:15.678437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:15.678437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:15.678437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:15.678437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:15.678437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:15.678437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 386/764] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:15.678923 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb5ad2-24dd-4967-b56c-1310533345f4 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:15.705752 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf9d5e-2d8a-412e-9ed6-b0c69e8f0a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:15.706502 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf9d5e-2d8a-412e-9ed6-b0c69e8f0a50 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:15.706780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf9d5e-2d8a-412e-9ed6-b0c69e8f0a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:15.707238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf9d5e-2d8a-412e-9ed6-b0c69e8f0a50 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:15.707775 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf9d5e-2d8a-412e-9ed6-b0c69e8f0a50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:15.708110 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf9d5e-2d8a-412e-9ed6-b0c69e8f0a50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:15.709482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf9d5e-2d8a-412e-9ed6-b0c69e8f0a50 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:15.721560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 379/765] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:15.722282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf9d5e-2d8a-412e-9ed6-b0c69e8f0a50 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:15.732807 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:15.772978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:15.795408 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb5ad2-24dd-4967-b56c-1310533345f4 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:15.830340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1cb1b9-b53b-40dc-aff2-e63f9bfcbeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:15.831066 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1cb1b9-b53b-40dc-aff2-e63f9bfcbeb5 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:15.831349 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1cb1b9-b53b-40dc-aff2-e63f9bfcbeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:15.831622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1cb1b9-b53b-40dc-aff2-e63f9bfcbeb5 None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:15.832382 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1cb1b9-b53b-40dc-aff2-e63f9bfcbeb5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:15.832692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1cb1b9-b53b-40dc-aff2-e63f9bfcbeb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:15.834058 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1cb1b9-b53b-40dc-aff2-e63f9bfcbeb5 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:15.835527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf9d5e-2d8a-412e-9ed6-b0c69e8f0a50 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:15.839838 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 387/766] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Jun 6 10:25:15 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:15.840690 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff46e7-8403-48fb-846f-3b13ee251f66 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:15.840978 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff46e7-8403-48fb-846f-3b13ee251f66 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:15.841262 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff46e7-8403-48fb-846f-3b13ee251f66 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:15.847406 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 388/767] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:15.847406 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff46e7-8403-48fb-846f-3b13ee251f66 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:15.879192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c427ad-1539-48e2-9b78-9a1df780dd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:15.880187 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c427ad-1539-48e2-9b78-9a1df780dd4e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:15.880731 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c427ad-1539-48e2-9b78-9a1df780dd4e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:15.881014 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c427ad-1539-48e2-9b78-9a1df780dd4e None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:15.881890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c427ad-1539-48e2-9b78-9a1df780dd4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:15.882199 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c427ad-1539-48e2-9b78-9a1df780dd4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:15.883609 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c427ad-1539-48e2-9b78-9a1df780dd4e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:15.904690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 380/768] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:25:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:15.904690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c427ad-1539-48e2-9b78-9a1df780dd4e None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:15.951161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff46e7-8403-48fb-846f-3b13ee251f66 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:15.959735 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:15.988011 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7f161-f72a-499e-9892-e95590e8b606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:15.988817 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7f161-f72a-499e-9892-e95590e8b606 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:15.989073 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7f161-f72a-499e-9892-e95590e8b606 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:15.989338 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7f161-f72a-499e-9892-e95590e8b606 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:15.997008 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7f161-f72a-499e-9892-e95590e8b606 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:15.998562 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7f161-f72a-499e-9892-e95590e8b606 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:16.012859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-0cd7f161-f72a-499e-9892-e95590e8b606 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:16.012859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:16.012859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:16.012859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:16.012859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:16.012859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:16.012859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:16.012859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:16.012859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:16.012859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:16.012859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:16.012859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:16.012859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:16.012859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:16.012859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:16.013756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:16.013756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:16.013756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:16.013756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:16.013756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:16.013756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:16.013756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:16.013756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:16.013756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:16.013756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:16.013756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:16.013756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:16.013756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:16.013756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:16.013756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:16.013756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:16.013756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:16.014670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:16.014670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:16.014670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:16.014670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:16.014670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:16.014670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:16.014670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:16.014670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:16.014670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 389/769] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:16.015680 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7367e8-f66e-4165-91ae-d9eafd7e7a9e None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:16.015680 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7367e8-f66e-4165-91ae-d9eafd7e7a9e None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:16.016065 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7367e8-f66e-4165-91ae-d9eafd7e7a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:16.030267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c427ad-1539-48e2-9b78-9a1df780dd4e None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:16.033143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-da7367e8-f66e-4165-91ae-d9eafd7e7a9e None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:16.034373 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-da7367e8-f66e-4165-91ae-d9eafd7e7a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:16.037962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:16.066631 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172715d-2577-489c-8eb6-2a83254c7a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:16.067305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172715d-2577-489c-8eb6-2a83254c7a8c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:16.067590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172715d-2577-489c-8eb6-2a83254c7a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:16.067893 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172715d-2577-489c-8eb6-2a83254c7a8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:16.068685 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b172715d-2577-489c-8eb6-2a83254c7a8c None admin] RBAC: Adding query filter params (interface=public, service_id=2f9b7000869a453895811fe70341ee3c, region_id=RegionOne) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:16.068996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b172715d-2577-489c-8eb6-2a83254c7a8c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:16.070380 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b172715d-2577-489c-8eb6-2a83254c7a8c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:16.082980 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 381/770] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jun 6 10:25:15 2022] GET /identity/v3/endpoints?service_id=2f9b7000869a453895811fe70341ee3c&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:16.082980 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172715d-2577-489c-8eb6-2a83254c7a8c None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:16.115515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 390/771] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:16.116693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7367e8-f66e-4165-91ae-d9eafd7e7a9e None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:16.202505 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172715d-2577-489c-8eb6-2a83254c7a8c None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:16.229489 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:16.236536 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7367e8-f66e-4165-91ae-d9eafd7e7a9e None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:16.238058 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d85ee7-8d18-415a-90cb-effacec11300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:16.238712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d85ee7-8d18-415a-90cb-effacec11300 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:16.238996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d85ee7-8d18-415a-90cb-effacec11300 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:16.239270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d85ee7-8d18-415a-90cb-effacec11300 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:16.240133 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d85ee7-8d18-415a-90cb-effacec11300 None admin] RBAC: Adding query filter params (role.id=0f929204eb7b4ea8859338ed2d3ff75b, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=fede9e52ad274285b09d748a684ede38) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:16.240442 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d85ee7-8d18-415a-90cb-effacec11300 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:16.241872 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d85ee7-8d18-415a-90cb-effacec11300 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:16.255816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 382/772] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:25:16 2022] GET /identity/v3/role_assignments?user.id=fede9e52ad274285b09d748a684ede38&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=0f929204eb7b4ea8859338ed2d3ff75b => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:16.255816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d85ee7-8d18-415a-90cb-effacec11300 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:16.290899 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:16.296477 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5507ad8-7031-4d9b-a89b-8a2ca4066cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:16.297154 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5507ad8-7031-4d9b-a89b-8a2ca4066cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:16.297434 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5507ad8-7031-4d9b-a89b-8a2ca4066cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:16.297706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5507ad8-7031-4d9b-a89b-8a2ca4066cd3 None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:16.300633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5507ad8-7031-4d9b-a89b-8a2ca4066cd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:16.300947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5507ad8-7031-4d9b-a89b-8a2ca4066cd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:16.303036 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5507ad8-7031-4d9b-a89b-8a2ca4066cd3 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:16.320887 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 391/773] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:25:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:16.322572 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5507ad8-7031-4d9b-a89b-8a2ca4066cd3 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:16.373057 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d85ee7-8d18-415a-90cb-effacec11300 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:16.422292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91926a9-57fe-4e24-85a4-e050a71931ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:16.422976 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91926a9-57fe-4e24-85a4-e050a71931ba None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:16.423260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91926a9-57fe-4e24-85a4-e050a71931ba None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:16.423536 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91926a9-57fe-4e24-85a4-e050a71931ba None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:16.424304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91926a9-57fe-4e24-85a4-e050a71931ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:16.425712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91926a9-57fe-4e24-85a4-e050a71931ba None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:16.433714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 383/774] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jun 6 10:25:16 2022] GET /identity/v3/services => generated 772 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:16.434748 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91926a9-57fe-4e24-85a4-e050a71931ba None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:16.457166 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5507ad8-7031-4d9b-a89b-8a2ca4066cd3 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:16.498670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0f7f9-14c8-4d6c-a251-d3e3443dbf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:16.499474 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0f7f9-14c8-4d6c-a251-d3e3443dbf13 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:16.499755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0f7f9-14c8-4d6c-a251-d3e3443dbf13 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:16.500058 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0f7f9-14c8-4d6c-a251-d3e3443dbf13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:16.512609 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc0f7f9-14c8-4d6c-a251-d3e3443dbf13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:25:16.521692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-dfc0f7f9-14c8-4d6c-a251-d3e3443dbf13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:25:16.521692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:16.521692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:16.521692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:16.521692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:16.521692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:16.521692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:16.521692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:16.521692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:16.521692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:16.521692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:16.521692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:16.521692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:25:16.521692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:25:16.521692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:25:16.521692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:25:16.522671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:25:16.522671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:25:16.522671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:25:16.522671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:25:16.522671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:16.522671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:16.522671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:16.522671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:16.522671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:16.522671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:16.522671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:16.522671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:16.522671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:16.522671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:16.522671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:16.522671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:16.523739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:16.523739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:16.523739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:16.523739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:16.523739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:25:16.523739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:25:16.523739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:25:16.523739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:25:16.523739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:25:16.523739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:25:16.523739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:25:16.523739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:16.523739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 392/775] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:25:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:16.524946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e4838-af4b-4d53-8283-aaa923d9e749 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:16.525117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e4838-af4b-4d53-8283-aaa923d9e749 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:16.529799 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e4838-af4b-4d53-8283-aaa923d9e749 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:16.529799 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 393/776] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:16.531878 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e4838-af4b-4d53-8283-aaa923d9e749 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:16.559586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:16.570959 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:16.576723 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91926a9-57fe-4e24-85a4-e050a71931ba None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:16.620792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3f930-8806-4efe-a6ad-37066d63d385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:16.621728 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3f930-8806-4efe-a6ad-37066d63d385 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:16.622110 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3f930-8806-4efe-a6ad-37066d63d385 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:16.622578 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3f930-8806-4efe-a6ad-37066d63d385 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:16.623531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3f930-8806-4efe-a6ad-37066d63d385 None admin] RBAC: Adding query filter params (role.id=95eccd30e39b4105ac6e8c0d9f973951, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=e9e358afe0974c0c8bd53104ebbff445) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:16.624010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3f930-8806-4efe-a6ad-37066d63d385 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:16.625586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3f930-8806-4efe-a6ad-37066d63d385 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:16.633487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 384/777] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:25:16 2022] GET /identity/v3/role_assignments?user.id=e9e358afe0974c0c8bd53104ebbff445&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=95eccd30e39b4105ac6e8c0d9f973951 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:16.634942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6363a-8ee9-4cb4-aee6-30c40715f811 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:16.635334 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6363a-8ee9-4cb4-aee6-30c40715f811 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:16.635709 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6363a-8ee9-4cb4-aee6-30c40715f811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:16.666624 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-fce6363a-8ee9-4cb4-aee6-30c40715f811 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:16.669268 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fce6363a-8ee9-4cb4-aee6-30c40715f811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:16.692017 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e4838-af4b-4d53-8283-aaa923d9e749 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:16.734315 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf0d1c-54ce-41c6-80d8-918c5c1de4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:16.735412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf0d1c-54ce-41c6-80d8-918c5c1de4a9 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:16.735884 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf0d1c-54ce-41c6-80d8-918c5c1de4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:16.736302 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf0d1c-54ce-41c6-80d8-918c5c1de4a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:16.737788 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf0d1c-54ce-41c6-80d8-918c5c1de4a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:16.738280 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf0d1c-54ce-41c6-80d8-918c5c1de4a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:16.740099 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf0d1c-54ce-41c6-80d8-918c5c1de4a9 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:16.756186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 394/778] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:25:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:16.760093 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 385/779] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:16.760762 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf0d1c-54ce-41c6-80d8-918c5c1de4a9 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:16.781430 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6363a-8ee9-4cb4-aee6-30c40715f811 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:16.874853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:16.910424 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf0d1c-54ce-41c6-80d8-918c5c1de4a9 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:16.938733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:16.954247 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b8e02-0795-4f37-81a5-07bb282f29d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:16.955086 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b8e02-0795-4f37-81a5-07bb282f29d9 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:16.955397 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b8e02-0795-4f37-81a5-07bb282f29d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:16.956677 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b8e02-0795-4f37-81a5-07bb282f29d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:16.960517 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6363a-8ee9-4cb4-aee6-30c40715f811 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:16.970912 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b8e02-0795-4f37-81a5-07bb282f29d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:16.975828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b8e02-0795-4f37-81a5-07bb282f29d9 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:16.993256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-1e4b8e02-0795-4f37-81a5-07bb282f29d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:16.993256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:16.993256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:16.993256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:16.993256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:16.993256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:16.993256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:16.993256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:16.993256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:16.993256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:16.993256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:16.993256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:16.993256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:16.993256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:16.993256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:16.994161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:16.994161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:16.994161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:16.994161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:16.994161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:16.994161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:16.994161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:16.994161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:16.994161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:16.994161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:16.994161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:16.994161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:16.994161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:16.994161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:16.994161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:16.994161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:16.994161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:16.995054 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:16.995054 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:16.995054 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:16.995054 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:16.995717 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 395/780] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:17.000589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b8e02-0795-4f37-81a5-07bb282f29d9 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:17.038781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb0769-7e49-4198-b65a-e8d3a2d301b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:17.039640 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb0769-7e49-4198-b65a-e8d3a2d301b3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:17.039999 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb0769-7e49-4198-b65a-e8d3a2d301b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:17.040277 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb0769-7e49-4198-b65a-e8d3a2d301b3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:17.041250 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb0769-7e49-4198-b65a-e8d3a2d301b3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:17.042633 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb0769-7e49-4198-b65a-e8d3a2d301b3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:17.056022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-a5eb0769-7e49-4198-b65a-e8d3a2d301b3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 10:25:17.056022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:17.056022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:17.056022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:17.056022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:17.056022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:17.056022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:17.056022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:17.056022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 10:25:17.056022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:17.056022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:17.056022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:17.056022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:17.056022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:17.056022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:17.056022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:17.056022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:17.057022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:17.057022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:17.057022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:17.057022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:17.057022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:17.057022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:17.057022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:17.057022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:17.057022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:17.057022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:17.057022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:17.057022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:17.057022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:17.057022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:17.057022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:17.057022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:17.057862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:17.057862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:17.057862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:17.057862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:17.057862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:17.057862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:17.057862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:17.057862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:17.057862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:17.057862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:17.057862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:17.057862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:17.057862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 10:25:17.057862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:17.057862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 386/781] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:16 2022] GET /identity/v3/services/network => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:17.063835 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb0769-7e49-4198-b65a-e8d3a2d301b3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:17.208654 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb0769-7e49-4198-b65a-e8d3a2d301b3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:17.211841 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b8e02-0795-4f37-81a5-07bb282f29d9 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:17.260147 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:17.270510 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cebaf-c386-451a-86bd-86c8f4c9900a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:17.271179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cebaf-c386-451a-86bd-86c8f4c9900a None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:17.271463 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cebaf-c386-451a-86bd-86c8f4c9900a None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:17.272429 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb763f07-f004-467d-91a9-b78f5f73aeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:17.273085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb763f07-f004-467d-91a9-b78f5f73aeee None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:17.273616 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb763f07-f004-467d-91a9-b78f5f73aeee None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:17.273886 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb763f07-f004-467d-91a9-b78f5f73aeee None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:17.274593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb763f07-f004-467d-91a9-b78f5f73aeee None admin] RBAC: Adding query filter params (name=network) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:17.274897 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb763f07-f004-467d-91a9-b78f5f73aeee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:17.276294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb763f07-f004-467d-91a9-b78f5f73aeee None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:17.277286 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cebaf-c386-451a-86bd-86c8f4c9900a None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:17.278025 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cebaf-c386-451a-86bd-86c8f4c9900a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:17.278336 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cebaf-c386-451a-86bd-86c8f4c9900a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:17.279680 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cebaf-c386-451a-86bd-86c8f4c9900a None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:17.289993 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 387/782] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jun 6 10:25:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:17.295102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb763f07-f004-467d-91a9-b78f5f73aeee None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:17.300347 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 396/783] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:17.302375 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cebaf-c386-451a-86bd-86c8f4c9900a None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:17.335469 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:17.436497 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cebaf-c386-451a-86bd-86c8f4c9900a None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:17.479958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41111cb-ab60-43bf-839b-dec04298fc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:17.479958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41111cb-ab60-43bf-839b-dec04298fc82 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:17.479958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41111cb-ab60-43bf-839b-dec04298fc82 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:17.479958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41111cb-ab60-43bf-839b-dec04298fc82 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:17.485503 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41111cb-ab60-43bf-839b-dec04298fc82 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:17.487757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41111cb-ab60-43bf-839b-dec04298fc82 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:17.490337 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb763f07-f004-467d-91a9-b78f5f73aeee None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:17.504021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-e41111cb-ab60-43bf-839b-dec04298fc82 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 10:25:17.504021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:17.504021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:17.504021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:17.504021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:17.504021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:17.504021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:17.504021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:17.504021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:17.504021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:17.504021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:17.504021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:17.504021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:25:17.504021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:25:17.504021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:25:17.504021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:25:17.505022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:17.505022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:17.505022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:17.505022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:17.505022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:17.505022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:17.505022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:17.505022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:17.505022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:17.505022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:17.505022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:17.505022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:17.505022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:17.505022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:17.505022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:17.505022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:17.505022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:17.505022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:17.505963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:17.505963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:17.505963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:25:17.505963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:25:17.505963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:25:17.505963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:25:17.505963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:25:17.505963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:25:17.505963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 10:25:17.505963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:17.505963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 397/784] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jun 6 10:25:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:17.512652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41111cb-ab60-43bf-839b-dec04298fc82 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:17.580232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:17.601852 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd182c-9b50-4460-b276-83ca16a6413d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:17.602533 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd182c-9b50-4460-b276-83ca16a6413d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:17.602816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd182c-9b50-4460-b276-83ca16a6413d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:17.603090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd182c-9b50-4460-b276-83ca16a6413d None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:17.603819 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd182c-9b50-4460-b276-83ca16a6413d None admin] RBAC: Adding query filter params (type=network) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:17.604130 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd182c-9b50-4460-b276-83ca16a6413d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:17.605521 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd182c-9b50-4460-b276-83ca16a6413d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:17.616967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 388/785] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jun 6 10:25:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:17.621419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd182c-9b50-4460-b276-83ca16a6413d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:17.656973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41111cb-ab60-43bf-839b-dec04298fc82 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:17.726957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f75d3-c672-4514-aa4e-b16e433adc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:17.727629 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f75d3-c672-4514-aa4e-b16e433adc57 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:17.727937 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f75d3-c672-4514-aa4e-b16e433adc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:17.728210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f75d3-c672-4514-aa4e-b16e433adc57 None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:17.729042 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f75d3-c672-4514-aa4e-b16e433adc57 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:17.729347 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f75d3-c672-4514-aa4e-b16e433adc57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:17.730979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f75d3-c672-4514-aa4e-b16e433adc57 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:17.771089 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 398/786] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jun 6 10:25:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:17.776350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f75d3-c672-4514-aa4e-b16e433adc57 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:17.787813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:17.808799 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd182c-9b50-4460-b276-83ca16a6413d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:17.843477 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff427b-dac1-46fc-89c0-400f62d761ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:17.844475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff427b-dac1-46fc-89c0-400f62d761ad None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:17.844989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff427b-dac1-46fc-89c0-400f62d761ad None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:17.845473 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff427b-dac1-46fc-89c0-400f62d761ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:17.847172 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff427b-dac1-46fc-89c0-400f62d761ad None admin] RBAC: Adding query filter params (interface=public, service_id=a470520270e342a8afb1152668c91311, region_id=RegionOne) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:17.847686 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff427b-dac1-46fc-89c0-400f62d761ad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:17.849281 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff427b-dac1-46fc-89c0-400f62d761ad None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:17.854919 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 389/787] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jun 6 10:25:17 2022] GET /identity/v3/endpoints?service_id=a470520270e342a8afb1152668c91311&interface=public®ion_id=RegionOne => generated 194 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:17.863814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff427b-dac1-46fc-89c0-400f62d761ad None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:17.887631 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f75d3-c672-4514-aa4e-b16e433adc57 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:17.904234 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:17.928946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf883fd7-8e2a-409c-9411-107b923f7ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:17.929600 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf883fd7-8e2a-409c-9411-107b923f7ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:17.930475 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf883fd7-8e2a-409c-9411-107b923f7ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:17.930754 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf883fd7-8e2a-409c-9411-107b923f7ed7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:18.073958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf883fd7-8e2a-409c-9411-107b923f7ed7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:18.075586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf883fd7-8e2a-409c-9411-107b923f7ed7 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:18.095014 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff427b-dac1-46fc-89c0-400f62d761ad None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:18.166269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf979df6-c1ad-43b0-afc3-3495ce03d500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:18.166936 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf979df6-c1ad-43b0-afc3-3495ce03d500 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:18.167226 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf979df6-c1ad-43b0-afc3-3495ce03d500 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:18.167495 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf979df6-c1ad-43b0-afc3-3495ce03d500 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:18.168202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf979df6-c1ad-43b0-afc3-3495ce03d500 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:18.169574 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf979df6-c1ad-43b0-afc3-3495ce03d500 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:18.183278 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 390/788] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jun 6 10:25:17 2022] GET /identity/v3/services => generated 772 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:18.237065 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 399/789] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:25:17 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 06 10:25:18.242133 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf979df6-c1ad-43b0-afc3-3495ce03d500 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:18.276322 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:18.398638 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf979df6-c1ad-43b0-afc3-3495ce03d500 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:18.444262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f931d9-66be-4067-9514-4011dadfedab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:18.445514 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f931d9-66be-4067-9514-4011dadfedab None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:18.445795 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f931d9-66be-4067-9514-4011dadfedab None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:18.446067 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f931d9-66be-4067-9514-4011dadfedab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:18.458158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f931d9-66be-4067-9514-4011dadfedab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:18.459669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f931d9-66be-4067-9514-4011dadfedab None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:18.477378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-a9f931d9-66be-4067-9514-4011dadfedab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:18.477378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:18.477378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:18.477378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:18.477378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:18.477378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:18.477378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:18.477378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:18.477378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:18.477378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:18.477378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:18.477378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:18.477378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:18.477378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:18.477378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:18.478307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:18.478307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:18.478307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:18.478307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:18.478307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:18.478307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:18.478307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:18.478307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:18.478307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:18.478307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:18.478307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:18.478307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:18.478307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:18.478307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:18.478307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:18.478307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:18.478307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:18.479168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:18.479168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:18.479168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:18.479168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:18.479168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 391/790] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:18.484907 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf883fd7-8e2a-409c-9411-107b923f7ed7 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:18.498490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:18.593266 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf883fd7-8e2a-409c-9411-107b923f7ed7 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:18.635437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20e28f-66cd-419b-be4a-a786e71e91c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:18.636481 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20e28f-66cd-419b-be4a-a786e71e91c3 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:18.636977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20e28f-66cd-419b-be4a-a786e71e91c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:18.637446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20e28f-66cd-419b-be4a-a786e71e91c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:18.638373 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20e28f-66cd-419b-be4a-a786e71e91c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:18.638887 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20e28f-66cd-419b-be4a-a786e71e91c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:18.640482 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20e28f-66cd-419b-be4a-a786e71e91c3 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:18.658001 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 400/791] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:18.664540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f931d9-66be-4067-9514-4011dadfedab None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:18.748549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:18.772222 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72e271-7461-452e-b2cf-33490cad2b56 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:18.773007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72e271-7461-452e-b2cf-33490cad2b56 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:18.773742 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72e271-7461-452e-b2cf-33490cad2b56 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:18.775618 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 401/792] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:25:18.780931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641350e6-beff-48a3-b747-4e97cd90265d None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:18.781559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641350e6-beff-48a3-b747-4e97cd90265d None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:18.782067 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641350e6-beff-48a3-b747-4e97cd90265d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:18.801641 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-641350e6-beff-48a3-b747-4e97cd90265d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:18.803194 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-641350e6-beff-48a3-b747-4e97cd90265d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:18.809447 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f931d9-66be-4067-9514-4011dadfedab None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:18.884435 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f64b776-a505-418b-871e-a0bf4cf74aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:18.885246 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64b776-a505-418b-871e-a0bf4cf74aff None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:18.886743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64b776-a505-418b-871e-a0bf4cf74aff None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:18.886979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64b776-a505-418b-871e-a0bf4cf74aff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:18.899595 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f64b776-a505-418b-871e-a0bf4cf74aff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:18.902132 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f64b776-a505-418b-871e-a0bf4cf74aff None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:18.931569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-7f64b776-a505-418b-871e-a0bf4cf74aff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:18.931569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:18.931569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:18.931569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:18.931569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:18.931569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:18.931569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:18.931569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:18.931569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:18.931569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:18.931569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:18.931569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:18.931569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:18.931569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:18.931569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:18.932552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:18.932552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:18.932552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:18.932552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:18.932552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:18.932552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:18.932552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:18.932552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:18.932552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:18.932552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:18.932552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:18.932552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:18.932552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:18.932552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:18.932552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:18.932552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:18.932552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:18.933434 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:18.933434 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:18.933434 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:18.933434 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:18.933434 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:18.933434 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:18.933434 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:18.933434 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:18.934567 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:18.935657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 392/793] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:18.942652 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f64b776-a505-418b-871e-a0bf4cf74aff None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:18.945180 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 402/794] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 10:25:19.003478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641350e6-beff-48a3-b747-4e97cd90265d None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:19.110313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f64b776-a505-418b-871e-a0bf4cf74aff None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:19.136485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:19.149724 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6a50e-1226-4d37-8661-4a3dc1b8e0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:19.155870 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6a50e-1226-4d37-8661-4a3dc1b8e0af None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:19.155870 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6a50e-1226-4d37-8661-4a3dc1b8e0af None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:19.155870 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6a50e-1226-4d37-8661-4a3dc1b8e0af None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:19.155870 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6a50e-1226-4d37-8661-4a3dc1b8e0af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:19.155870 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6a50e-1226-4d37-8661-4a3dc1b8e0af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:19.155870 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6a50e-1226-4d37-8661-4a3dc1b8e0af None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:19.172547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641350e6-beff-48a3-b747-4e97cd90265d None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:19.178992 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 393/795] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:25:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:19.192134 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6a50e-1226-4d37-8661-4a3dc1b8e0af None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:19.222055 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda0500-f7b3-4df1-a2b0-67bbad85d203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:19.223223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda0500-f7b3-4df1-a2b0-67bbad85d203 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:19.223726 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda0500-f7b3-4df1-a2b0-67bbad85d203 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:19.224247 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda0500-f7b3-4df1-a2b0-67bbad85d203 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:19.246301 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda0500-f7b3-4df1-a2b0-67bbad85d203 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:25:19.254962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-4cda0500-f7b3-4df1-a2b0-67bbad85d203 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:25:19.254962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:19.254962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:19.254962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:19.254962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:19.254962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:19.254962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:19.254962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:19.254962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:19.254962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:19.254962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:19.254962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:19.254962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:25:19.254962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:25:19.254962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:25:19.254962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:25:19.255898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:25:19.255898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:25:19.255898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:25:19.255898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:25:19.255898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:19.255898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:19.255898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:19.255898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:19.255898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:19.255898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:19.255898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:19.255898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:19.255898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:19.255898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:19.255898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:19.255898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:19.256727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:19.256727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:19.256727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:19.256727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:19.256727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:25:19.256727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:25:19.256727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:25:19.256727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:25:19.256727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:25:19.256727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:25:19.256727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:25:19.256727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:19.274246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 403/796] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:25:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:19.279439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda0500-f7b3-4df1-a2b0-67bbad85d203 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:19.360054 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:19.362465 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6a50e-1226-4d37-8661-4a3dc1b8e0af None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:19.399956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449bd90-8201-4f09-8bab-f678142f8467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:19.400918 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449bd90-8201-4f09-8bab-f678142f8467 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:19.401395 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449bd90-8201-4f09-8bab-f678142f8467 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:19.401844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449bd90-8201-4f09-8bab-f678142f8467 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:19.402810 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7449bd90-8201-4f09-8bab-f678142f8467 None admin] RBAC: Adding query filter params (role.id=0f929204eb7b4ea8859338ed2d3ff75b, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=ed8a40e4f1034a8e99d0eb505e448db9) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:19.403263 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7449bd90-8201-4f09-8bab-f678142f8467 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:19.404838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7449bd90-8201-4f09-8bab-f678142f8467 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:19.410982 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda0500-f7b3-4df1-a2b0-67bbad85d203 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:19.419811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 394/797] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:25:19 2022] GET /identity/v3/role_assignments?user.id=ed8a40e4f1034a8e99d0eb505e448db9&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=0f929204eb7b4ea8859338ed2d3ff75b => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:19.419811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d406c-50bf-4038-bc97-63882bcda70e None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:19.419811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d406c-50bf-4038-bc97-63882bcda70e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:19.419811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d406c-50bf-4038-bc97-63882bcda70e None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:19.425342 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 395/798] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:19.426342 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dcff86-dd5b-4faf-88b3-652e43728a7f None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:19.426629 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dcff86-dd5b-4faf-88b3-652e43728a7f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:19.426909 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dcff86-dd5b-4faf-88b3-652e43728a7f None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:19.433204 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 396/799] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:19.434519 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481050e7-f5d6-4c7f-87a9-48c333b50ee7 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:19.434810 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481050e7-f5d6-4c7f-87a9-48c333b50ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:19.435091 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481050e7-f5d6-4c7f-87a9-48c333b50ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:19.454470 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-481050e7-f5d6-4c7f-87a9-48c333b50ee7 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:19.455727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-481050e7-f5d6-4c7f-87a9-48c333b50ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:19.460131 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c852f9b-0372-4de6-8604-adc03e17a45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:19.460951 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c852f9b-0372-4de6-8604-adc03e17a45f None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:19.461238 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c852f9b-0372-4de6-8604-adc03e17a45f None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:19.461513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c852f9b-0372-4de6-8604-adc03e17a45f None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:19.462275 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c852f9b-0372-4de6-8604-adc03e17a45f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:19.462582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c852f9b-0372-4de6-8604-adc03e17a45f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:19.464012 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c852f9b-0372-4de6-8604-adc03e17a45f None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:19.480038 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 404/800] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:25:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:19.481552 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53a846-dca1-484b-b3af-998ffb9396fa None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:19.481839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53a846-dca1-484b-b3af-998ffb9396fa None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:19.482190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53a846-dca1-484b-b3af-998ffb9396fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:19.501271 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-bb53a846-dca1-484b-b3af-998ffb9396fa None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:19.503222 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-bb53a846-dca1-484b-b3af-998ffb9396fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:19.517615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:19.552317 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 397/801] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:19.553350 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481050e7-f5d6-4c7f-87a9-48c333b50ee7 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:19.603737 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 405/802] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:19.604729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53a846-dca1-484b-b3af-998ffb9396fa None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:19.663011 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:19.704816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481050e7-f5d6-4c7f-87a9-48c333b50ee7 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:19.733570 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53a846-dca1-484b-b3af-998ffb9396fa None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:19.749425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18136571-8483-40bc-aa14-552884da9ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:19.750840 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18136571-8483-40bc-aa14-552884da9ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:19.751303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18136571-8483-40bc-aa14-552884da9ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:19.752030 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18136571-8483-40bc-aa14-552884da9ea1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:19.766021 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18136571-8483-40bc-aa14-552884da9ea1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:19.767757 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18136571-8483-40bc-aa14-552884da9ea1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:19.774696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c2b48-0d69-481e-ab98-63f69d2ce496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:19.775581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c2b48-0d69-481e-ab98-63f69d2ce496 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:19.775961 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c2b48-0d69-481e-ab98-63f69d2ce496 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:19.776526 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c2b48-0d69-481e-ab98-63f69d2ce496 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:19.788791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-18136571-8483-40bc-aa14-552884da9ea1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:19.788791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:19.788791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:19.788791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:19.788791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:19.788791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:19.788791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:19.788791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:19.788791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:19.788791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:19.788791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:19.788791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:19.788791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:19.788791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:19.788791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:19.789753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:19.789753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:19.789753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:19.789753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:19.789753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:19.789753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:19.789753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:19.789753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:19.789753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:19.789753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:19.789753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:19.789753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:19.789753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:19.789753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:19.789753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:19.789753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:19.789753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:19.790638 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:19.790638 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:19.790638 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:19.790638 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:19.790859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c2b48-0d69-481e-ab98-63f69d2ce496 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:25:19.793209 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 398/803] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:19.794303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18136571-8483-40bc-aa14-552884da9ea1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:19.800548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-101c2b48-0d69-481e-ab98-63f69d2ce496 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:25:19.800548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:19.800548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:19.800548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:19.800548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:19.800548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:19.800548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:19.800548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:19.800548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:19.800548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:19.800548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:19.800548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:19.800548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:25:19.800548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:25:19.800548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:25:19.800548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:25:19.801490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:25:19.801490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:25:19.801490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:25:19.801490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:25:19.801490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:19.801490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:19.801490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:19.801490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:19.801490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:19.801490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:19.801490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:19.801490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:19.801490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:19.801490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:19.801490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:19.801490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:19.802350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:19.802350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:19.802350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:19.802350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:19.802350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:25:19.802350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:25:19.802350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:25:19.802350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:25:19.802350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:25:19.802350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:25:19.802350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:25:19.802350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:19.806093 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 406/804] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:25:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:19.807048 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c2b48-0d69-481e-ab98-63f69d2ce496 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:19.847625 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:19.939257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18136571-8483-40bc-aa14-552884da9ea1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:19.940985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c2b48-0d69-481e-ab98-63f69d2ce496 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:19.955185 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:19.979859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aae6c0-2bfe-48ac-845a-9c5ae8ac9faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:19.980579 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aae6c0-2bfe-48ac-845a-9c5ae8ac9faa None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:19.980807 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aae6c0-2bfe-48ac-845a-9c5ae8ac9faa None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:19.981088 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aae6c0-2bfe-48ac-845a-9c5ae8ac9faa None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:19.981807 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aae6c0-2bfe-48ac-845a-9c5ae8ac9faa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:19.982213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aae6c0-2bfe-48ac-845a-9c5ae8ac9faa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:19.983583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aae6c0-2bfe-48ac-845a-9c5ae8ac9faa None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:19.986187 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e5d70-4418-428d-aef6-9026e9c3db61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:19.987840 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e5d70-4418-428d-aef6-9026e9c3db61 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:19.988328 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e5d70-4418-428d-aef6-9026e9c3db61 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:19.988802 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e5d70-4418-428d-aef6-9026e9c3db61 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:19.989654 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e5d70-4418-428d-aef6-9026e9c3db61 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:19.991229 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e5d70-4418-428d-aef6-9026e9c3db61 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:20.001697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 407/805] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:20.002891 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aae6c0-2bfe-48ac-845a-9c5ae8ac9faa None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:20.005559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-c71e5d70-4418-428d-aef6-9026e9c3db61 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 10:25:20.005559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:20.005559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:20.005559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:20.005559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:20.005559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:20.005559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:20.005559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:20.005559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 10:25:20.005559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:20.005559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:20.005559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:20.005559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:20.005559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:20.005559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:20.005559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:20.006499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:20.006499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:20.006499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:20.006499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:20.006499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:20.006499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:20.006499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:20.006499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:20.006499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:20.006499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:20.006499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:20.006499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:20.006499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:20.006499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:20.006499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:20.006499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:20.007364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:20.007364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:20.007364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:20.007364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:20.007364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:20.007364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:20.007364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:20.007364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:20.007364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:20.007364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:20.007364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:20.007364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:20.007364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:20.007364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 10:25:20.007364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:20.014962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 399/806] 10.222.0.42 () {56 vars in 1129 bytes} [Mon Jun 6 10:25:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:20.016231 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e5d70-4418-428d-aef6-9026e9c3db61 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:20.120102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:20.128234 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aae6c0-2bfe-48ac-845a-9c5ae8ac9faa None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:20.130278 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e5d70-4418-428d-aef6-9026e9c3db61 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:20.177861 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee90ec-b5b2-4ddb-b80c-66436a2e31dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:20.178795 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee90ec-b5b2-4ddb-b80c-66436a2e31dd None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:20.179085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee90ec-b5b2-4ddb-b80c-66436a2e31dd None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:20.179365 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee90ec-b5b2-4ddb-b80c-66436a2e31dd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:20.180404 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89060461-fdbd-4c77-9e39-52ad54c1c5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:20.181286 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89060461-fdbd-4c77-9e39-52ad54c1c5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:20.181767 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89060461-fdbd-4c77-9e39-52ad54c1c5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:20.182239 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89060461-fdbd-4c77-9e39-52ad54c1c5e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:20.183190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89060461-fdbd-4c77-9e39-52ad54c1c5e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:20.183706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89060461-fdbd-4c77-9e39-52ad54c1c5e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:20.185355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89060461-fdbd-4c77-9e39-52ad54c1c5e9 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:20.188117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee90ec-b5b2-4ddb-b80c-66436a2e31dd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:20.189632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee90ec-b5b2-4ddb-b80c-66436a2e31dd None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:20.195009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 408/807] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jun 6 10:25:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:20.196070 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89060461-fdbd-4c77-9e39-52ad54c1c5e9 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:20.209956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-a7ee90ec-b5b2-4ddb-b80c-66436a2e31dd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 10:25:20.209956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:20.209956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:20.209956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:20.209956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:20.209956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:20.209956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:20.209956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:20.209956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:20.209956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:20.209956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:20.209956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:20.209956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:25:20.209956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:25:20.209956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:25:20.209956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:25:20.210956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:20.210956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:20.210956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:20.210956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:20.210956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:20.210956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:20.210956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:20.210956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:20.210956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:20.210956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:20.210956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:20.210956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:20.210956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:20.210956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:20.210956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:20.210956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:20.210956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:20.210956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:20.211995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:20.211995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:20.211995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:25:20.211995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:25:20.211995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:25:20.211995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:25:20.211995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:25:20.211995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:25:20.211995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 10:25:20.211995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:20.214626 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 400/808] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:25:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:20.215722 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee90ec-b5b2-4ddb-b80c-66436a2e31dd None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:20.241179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:20.329933 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89060461-fdbd-4c77-9e39-52ad54c1c5e9 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:20.355340 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee90ec-b5b2-4ddb-b80c-66436a2e31dd None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:20.382696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554d5f8-ea85-4e1c-b2e7-20f2c3f65846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:20.382696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554d5f8-ea85-4e1c-b2e7-20f2c3f65846 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:20.382696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554d5f8-ea85-4e1c-b2e7-20f2c3f65846 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:20.382696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554d5f8-ea85-4e1c-b2e7-20f2c3f65846 None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:20.383125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554d5f8-ea85-4e1c-b2e7-20f2c3f65846 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:20.383125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554d5f8-ea85-4e1c-b2e7-20f2c3f65846 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:20.383125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554d5f8-ea85-4e1c-b2e7-20f2c3f65846 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:20.394691 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 409/809] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Jun 6 10:25:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:20.394691 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554d5f8-ea85-4e1c-b2e7-20f2c3f65846 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:20.407421 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7af22b-9ce8-47a2-bbd0-e35ccd287f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:20.409229 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7af22b-9ce8-47a2-bbd0-e35ccd287f78 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:20.409715 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7af22b-9ce8-47a2-bbd0-e35ccd287f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:20.410298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7af22b-9ce8-47a2-bbd0-e35ccd287f78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:20.422253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7af22b-9ce8-47a2-bbd0-e35ccd287f78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:20.424155 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7af22b-9ce8-47a2-bbd0-e35ccd287f78 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:20.443884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-7f7af22b-9ce8-47a2-bbd0-e35ccd287f78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:20.443884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:20.443884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:20.443884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:20.443884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:20.443884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:20.443884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:20.443884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:20.443884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:20.443884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:20.443884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:20.443884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:20.443884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:20.443884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:20.443884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:20.444794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:20.444794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:20.444794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:20.444794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:20.444794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:20.444794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:20.444794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:20.444794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:20.444794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:20.444794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:20.444794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:20.444794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:20.444794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:20.444794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:20.444794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:20.444794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:20.444794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:20.445702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:20.445702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:20.445702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:20.445702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:20.452190 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 401/810] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:20.452996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7af22b-9ce8-47a2-bbd0-e35ccd287f78 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:20.466485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:20.507591 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:20.516034 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554d5f8-ea85-4e1c-b2e7-20f2c3f65846 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:20.555202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07fc5e-f0aa-4a8b-bf9e-47a4a2523400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:20.556372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07fc5e-f0aa-4a8b-bf9e-47a4a2523400 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:20.556855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07fc5e-f0aa-4a8b-bf9e-47a4a2523400 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:20.557346 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07fc5e-f0aa-4a8b-bf9e-47a4a2523400 None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:20.558468 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07fc5e-f0aa-4a8b-bf9e-47a4a2523400 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:20.558993 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07fc5e-f0aa-4a8b-bf9e-47a4a2523400 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:20.560689 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07fc5e-f0aa-4a8b-bf9e-47a4a2523400 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:20.582478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 410/811] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Jun 6 10:25:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:20.582655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7af22b-9ce8-47a2-bbd0-e35ccd287f78 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:20.583681 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07fc5e-f0aa-4a8b-bf9e-47a4a2523400 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:20.624079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1fd5b-2b42-4af2-b4a6-3ee7eb3bd43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:20.625157 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1fd5b-2b42-4af2-b4a6-3ee7eb3bd43c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:20.625643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1fd5b-2b42-4af2-b4a6-3ee7eb3bd43c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:20.626243 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1fd5b-2b42-4af2-b4a6-3ee7eb3bd43c None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:20.627881 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1fd5b-2b42-4af2-b4a6-3ee7eb3bd43c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:20.628707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1fd5b-2b42-4af2-b4a6-3ee7eb3bd43c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:20.630520 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1fd5b-2b42-4af2-b4a6-3ee7eb3bd43c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:20.641461 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 402/812] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Jun 6 10:25:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:20.643398 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1fd5b-2b42-4af2-b4a6-3ee7eb3bd43c None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:20.726297 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07fc5e-f0aa-4a8b-bf9e-47a4a2523400 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:20.730605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:20.766895 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1fd5b-2b42-4af2-b4a6-3ee7eb3bd43c None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:20.769872 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8fc54-2e37-4915-b34e-6cc9ae07583f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:20.771450 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8fc54-2e37-4915-b34e-6cc9ae07583f None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:20.771953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8fc54-2e37-4915-b34e-6cc9ae07583f None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:20.772337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8fc54-2e37-4915-b34e-6cc9ae07583f None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:20.773416 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8fc54-2e37-4915-b34e-6cc9ae07583f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:20.773835 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8fc54-2e37-4915-b34e-6cc9ae07583f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:20.775307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8fc54-2e37-4915-b34e-6cc9ae07583f None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:20.794874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 411/813] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:20.794874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8fc54-2e37-4915-b34e-6cc9ae07583f None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:20.811354 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127a008-a4cb-457c-b0c3-a81a81968df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:20.811680 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:20.813052 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127a008-a4cb-457c-b0c3-a81a81968df0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:20.813536 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127a008-a4cb-457c-b0c3-a81a81968df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:20.814061 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127a008-a4cb-457c-b0c3-a81a81968df0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:20.827982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127a008-a4cb-457c-b0c3-a81a81968df0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:20.829786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127a008-a4cb-457c-b0c3-a81a81968df0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:20.849668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-4127a008-a4cb-457c-b0c3-a81a81968df0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:20.849668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:20.849668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:20.849668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:20.849668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:20.849668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:20.849668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:20.849668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:20.849668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:20.849668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:20.849668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:20.849668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:20.849668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:20.849668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:20.849668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:20.850606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:20.850606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:20.850606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:20.850606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:20.850606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:20.850606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:20.850606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:20.850606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:20.850606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:20.850606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:20.850606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:20.850606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:20.850606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:20.850606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:20.850606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:20.850606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:20.850606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:20.851495 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:20.851495 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:20.851495 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:20.851495 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:20.853779 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 403/814] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:20.854777 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127a008-a4cb-457c-b0c3-a81a81968df0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:20.910683 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8fc54-2e37-4915-b34e-6cc9ae07583f None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:20.947305 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d867ec-9ec8-44f4-aea7-268a0b19eaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:20.947987 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d867ec-9ec8-44f4-aea7-268a0b19eaa3 None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:20.948269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d867ec-9ec8-44f4-aea7-268a0b19eaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:20.948545 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d867ec-9ec8-44f4-aea7-268a0b19eaa3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:20.949282 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d867ec-9ec8-44f4-aea7-268a0b19eaa3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:20.950652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d867ec-9ec8-44f4-aea7-268a0b19eaa3 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:20.966950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127a008-a4cb-457c-b0c3-a81a81968df0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:20.981058 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 412/815] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jun 6 10:25:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:25:20.982151 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d867ec-9ec8-44f4-aea7-268a0b19eaa3 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:20.994608 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:21.016434 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2edde-655d-4c87-9b4c-3a1e8953c039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:21.017555 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2edde-655d-4c87-9b4c-3a1e8953c039 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:21.018832 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2edde-655d-4c87-9b4c-3a1e8953c039 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:21.019700 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2edde-655d-4c87-9b4c-3a1e8953c039 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:21.032431 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2edde-655d-4c87-9b4c-3a1e8953c039 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:21.035128 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2edde-655d-4c87-9b4c-3a1e8953c039 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:21.056313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-e3a2edde-655d-4c87-9b4c-3a1e8953c039 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 10:25:21.056313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:21.056313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:21.056313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:21.056313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:21.056313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:21.056313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:21.056313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:21.056313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:21.056313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:21.056313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:21.056313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:21.056313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:25:21.056313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:25:21.056313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:25:21.056313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:25:21.057426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:21.057426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:21.057426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:21.057426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:21.057426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:21.057426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:21.057426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:21.057426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:21.057426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:21.057426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:21.057426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:21.057426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:21.057426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:21.057426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:21.057426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:21.057426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:21.057426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:21.057426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:21.058368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:21.058368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:21.058368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:25:21.058368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:25:21.058368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:25:21.058368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:25:21.058368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:25:21.058368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:25:21.058368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 10:25:21.058368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:21.063556 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 404/816] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jun 6 10:25:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:21.066390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8e848-ac91-46b9-becd-837be580397f None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:21.066844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8e848-ac91-46b9-becd-837be580397f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:21.067214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8e848-ac91-46b9-becd-837be580397f None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:21.070030 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 405/817] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:21.071097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8e848-ac91-46b9-becd-837be580397f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:21.083399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:21.131465 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d867ec-9ec8-44f4-aea7-268a0b19eaa3 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:21.174699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a2eed-1be1-4736-873e-1eb9af8cbe00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:21.175979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a2eed-1be1-4736-873e-1eb9af8cbe00 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:21.176481 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a2eed-1be1-4736-873e-1eb9af8cbe00 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:21.177016 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a2eed-1be1-4736-873e-1eb9af8cbe00 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:21.178102 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a2eed-1be1-4736-873e-1eb9af8cbe00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:21.178633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a2eed-1be1-4736-873e-1eb9af8cbe00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:21.180404 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a2eed-1be1-4736-873e-1eb9af8cbe00 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:21.199466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 413/818] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:21.201397 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc37a8a-b82a-4545-9810-cbf3520503dd None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:21.201991 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc37a8a-b82a-4545-9810-cbf3520503dd None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:21.202475 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc37a8a-b82a-4545-9810-cbf3520503dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:21.210807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8e848-ac91-46b9-becd-837be580397f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:21.222685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-9cc37a8a-b82a-4545-9810-cbf3520503dd None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:21.224205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-9cc37a8a-b82a-4545-9810-cbf3520503dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:21.256840 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d720ea-982a-43e1-ae21-b81bda664819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:21.257236 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d720ea-982a-43e1-ae21-b81bda664819 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:21.257514 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d720ea-982a-43e1-ae21-b81bda664819 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:21.257785 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d720ea-982a-43e1-ae21-b81bda664819 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:21.258619 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d720ea-982a-43e1-ae21-b81bda664819 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:21.258924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d720ea-982a-43e1-ae21-b81bda664819 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:21.260416 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d720ea-982a-43e1-ae21-b81bda664819 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:21.291869 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 406/819] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Jun 6 10:25:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:21.291869 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d720ea-982a-43e1-ae21-b81bda664819 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:21.313178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 414/820] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:21.314347 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc37a8a-b82a-4545-9810-cbf3520503dd None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:21.339865 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:21.387633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:21.436931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc37a8a-b82a-4545-9810-cbf3520503dd None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:21.442064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d720ea-982a-43e1-ae21-b81bda664819 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:21.479775 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae779cf1-88d4-4a43-acae-a225adb93ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:21.480732 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae779cf1-88d4-4a43-acae-a225adb93ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:21.480935 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae779cf1-88d4-4a43-acae-a225adb93ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:21.481200 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae779cf1-88d4-4a43-acae-a225adb93ea5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:21.486327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d5c2f-57f3-4c2a-b4a6-97b1f8531991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:21.488433 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d5c2f-57f3-4c2a-b4a6-97b1f8531991 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:21.489132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d5c2f-57f3-4c2a-b4a6-97b1f8531991 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:21.489729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d5c2f-57f3-4c2a-b4a6-97b1f8531991 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:21.490264 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae779cf1-88d4-4a43-acae-a225adb93ea5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:21.491866 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae779cf1-88d4-4a43-acae-a225adb93ea5 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:21.504135 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d5c2f-57f3-4c2a-b4a6-97b1f8531991 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:21.505893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d5c2f-57f3-4c2a-b4a6-97b1f8531991 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:21.507373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-ae779cf1-88d4-4a43-acae-a225adb93ea5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:21.507373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:21.507373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:21.507373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:21.507373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:21.507373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:21.507373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:21.507373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:21.507373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:21.507373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:21.507373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:21.507373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:21.507373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:21.507373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:21.507373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:21.508274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:21.508274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:21.508274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:21.508274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:21.508274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:21.508274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:21.508274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:21.508274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:21.508274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:21.508274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:21.508274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:21.508274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:21.508274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:21.508274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:21.508274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:21.508274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:21.508274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:21.509158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:21.509158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:21.509158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:21.509158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:21.509158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:21.509158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:21.509158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:21.509158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:21.513798 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 407/821] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:21.514812 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae779cf1-88d4-4a43-acae-a225adb93ea5 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:21.537440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-c54d5c2f-57f3-4c2a-b4a6-97b1f8531991 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:21.537440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:21.537440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:21.537440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:21.537440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:21.537440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:21.537440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:21.537440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:21.537440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:21.537440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:21.537440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:21.537440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:21.537440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:21.537440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:21.537440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:21.538395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:21.538395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:21.538395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:21.538395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:21.538395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:21.538395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:21.538395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:21.538395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:21.538395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:21.538395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:21.538395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:21.538395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:21.538395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:21.538395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:21.538395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:21.538395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:21.538395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:21.539275 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:21.539275 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:21.539275 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:21.539275 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:21.541993 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 415/822] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:21.543055 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d5c2f-57f3-4c2a-b4a6-97b1f8531991 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:21.622659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:21.630220 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae779cf1-88d4-4a43-acae-a225adb93ea5 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:21.669176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7facc-1f4d-4573-a3f0-b721a208233b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:21.670126 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7facc-1f4d-4573-a3f0-b721a208233b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:21.670483 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7facc-1f4d-4573-a3f0-b721a208233b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:21.670838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7facc-1f4d-4573-a3f0-b721a208233b None admin] PATH_INFO: `/v3/services/network` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:21.671773 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7facc-1f4d-4573-a3f0-b721a208233b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:21.673251 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7facc-1f4d-4573-a3f0-b721a208233b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:21.685270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d5c2f-57f3-4c2a-b4a6-97b1f8531991 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:21.686035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-07a7facc-1f4d-4573-a3f0-b721a208233b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 10:25:21.686035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:21.686035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:21.686035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:21.686035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:21.686035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:21.686035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:21.686035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:21.686035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 10:25:21.686035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:21.686035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:21.686035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:21.686035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:21.686035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:21.686035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:21.686035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:21.686035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:21.686970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:21.686970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:21.686970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:21.686970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:21.686970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:21.686970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:21.686970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:21.686970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:21.686970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:21.686970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:21.686970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:21.686970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:21.686970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:21.686970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:21.686970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:21.686970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:21.688073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:21.688073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:21.688073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:21.688073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:21.688073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:21.688073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:21.688073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:21.688073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:21.688073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:21.688073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:21.688073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:21.688073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:21.688073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 10:25:21.688073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:21.688073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 408/823] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:21 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:21.689846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7facc-1f4d-4573-a3f0-b721a208233b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:21.707726 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:21.740053 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310c208-288d-4b58-a5eb-09430444f230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:21.741199 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310c208-288d-4b58-a5eb-09430444f230 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:21.741662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310c208-288d-4b58-a5eb-09430444f230 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:21.742643 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310c208-288d-4b58-a5eb-09430444f230 None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:21.743890 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310c208-288d-4b58-a5eb-09430444f230 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:21.744341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310c208-288d-4b58-a5eb-09430444f230 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:21.746574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310c208-288d-4b58-a5eb-09430444f230 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:21.763638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 416/824] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:25:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:21.765267 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310c208-288d-4b58-a5eb-09430444f230 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:21.806239 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7facc-1f4d-4573-a3f0-b721a208233b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:21.844797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa5b15-e865-4d5d-9991-44d2458d7567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:21.845455 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa5b15-e865-4d5d-9991-44d2458d7567 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:21.845764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa5b15-e865-4d5d-9991-44d2458d7567 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:21.846616 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa5b15-e865-4d5d-9991-44d2458d7567 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:21.847688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa5b15-e865-4d5d-9991-44d2458d7567 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:21.848020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa5b15-e865-4d5d-9991-44d2458d7567 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:21.849383 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa5b15-e865-4d5d-9991-44d2458d7567 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:21.866371 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 409/825] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:21.866371 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa5b15-e865-4d5d-9991-44d2458d7567 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:21.887625 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310c208-288d-4b58-a5eb-09430444f230 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:21.888207 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:21.923407 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1214e-b832-4edf-bf9f-e3f880aa0724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:21.924734 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1214e-b832-4edf-bf9f-e3f880aa0724 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:21.925104 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1214e-b832-4edf-bf9f-e3f880aa0724 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:21.925427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1214e-b832-4edf-bf9f-e3f880aa0724 None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:21.926067 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1214e-b832-4edf-bf9f-e3f880aa0724 None admin] RBAC: Adding query filter params (name=network) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:21.926369 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1214e-b832-4edf-bf9f-e3f880aa0724 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:21.927715 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1214e-b832-4edf-bf9f-e3f880aa0724 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:21.937318 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 417/826] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jun 6 10:25:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:21.937318 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954ec8b-3e60-438b-bdf1-1efda9d4ffa4 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:21.937318 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954ec8b-3e60-438b-bdf1-1efda9d4ffa4 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:21.937318 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954ec8b-3e60-438b-bdf1-1efda9d4ffa4 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:21.937318 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 418/827] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:21.937318 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954ec8b-3e60-438b-bdf1-1efda9d4ffa4 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:21.974956 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:21.977122 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa5b15-e865-4d5d-9991-44d2458d7567 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:22.011991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc49a5-d40c-45de-a95f-6c836b34af09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:22.012869 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc49a5-d40c-45de-a95f-6c836b34af09 None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:22.013145 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc49a5-d40c-45de-a95f-6c836b34af09 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:22.013412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc49a5-d40c-45de-a95f-6c836b34af09 None admin] PATH_INFO: `/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/fede9e52ad274285b09d748a684ede38/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:22.015472 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc49a5-d40c-45de-a95f-6c836b34af09 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f082d7ed3694fd9998623f3d4a5fea9, user_id=fede9e52ad274285b09d748a684ede38, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:22.016855 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc49a5-d40c-45de-a95f-6c836b34af09 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:22.039808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 410/828] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:25:21 2022] PUT /identity/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/fede9e52ad274285b09d748a684ede38/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:25:22.039808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc49a5-d40c-45de-a95f-6c836b34af09 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:22.047237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954ec8b-3e60-438b-bdf1-1efda9d4ffa4 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:22.085928 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899918a-709d-4ac1-8f57-05e3ba6f3470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:22.086728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899918a-709d-4ac1-8f57-05e3ba6f3470 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:22.087007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899918a-709d-4ac1-8f57-05e3ba6f3470 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:22.087316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899918a-709d-4ac1-8f57-05e3ba6f3470 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:22.095133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899918a-709d-4ac1-8f57-05e3ba6f3470 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:22.097750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899918a-709d-4ac1-8f57-05e3ba6f3470 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:22.114709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-e899918a-709d-4ac1-8f57-05e3ba6f3470 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:22.114709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:22.114709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:22.114709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:22.114709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:22.114709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:22.114709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:22.114709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:22.114709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:22.114709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:22.114709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:22.114709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:22.114709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:22.114709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:22.114709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:22.115701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:22.115701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:22.115701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:22.115701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:22.115701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:22.115701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:22.115701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:22.115701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:22.115701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:22.115701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:22.115701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:22.115701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:22.115701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:22.115701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:22.115701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:22.115701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:22.115701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:22.116623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:22.116623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:22.116623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:22.116623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:22.116623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:22.116623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:22.116623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:22.116623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:22.120918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 419/829] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:22.122346 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e8c12-6e3c-4679-9061-ba04e75f5717 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:22.122666 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e8c12-6e3c-4679-9061-ba04e75f5717 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:22.122973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e8c12-6e3c-4679-9061-ba04e75f5717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:22.143732 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:22.147134 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-a70e8c12-6e3c-4679-9061-ba04e75f5717 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:22.149067 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a70e8c12-6e3c-4679-9061-ba04e75f5717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:22.175355 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc49a5-d40c-45de-a95f-6c836b34af09 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:22.224335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe3f9b-b2dc-4cac-995d-a13e8621b4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:22.225703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe3f9b-b2dc-4cac-995d-a13e8621b4d7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:22.226131 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe3f9b-b2dc-4cac-995d-a13e8621b4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:22.227091 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe3f9b-b2dc-4cac-995d-a13e8621b4d7 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:22.228354 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe3f9b-b2dc-4cac-995d-a13e8621b4d7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:22.229001 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe3f9b-b2dc-4cac-995d-a13e8621b4d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:22.231427 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe3f9b-b2dc-4cac-995d-a13e8621b4d7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:22.240369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 411/830] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jun 6 10:25:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:22.241482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe3f9b-b2dc-4cac-995d-a13e8621b4d7 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:22.247833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 420/831] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:22.248287 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e8c12-6e3c-4679-9061-ba04e75f5717 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:22.280325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:22.371171 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e8c12-6e3c-4679-9061-ba04e75f5717 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:22.391631 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe3f9b-b2dc-4cac-995d-a13e8621b4d7 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:22.422475 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c12549-8d8f-45c5-8229-934af3466041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:22.423388 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c12549-8d8f-45c5-8229-934af3466041 None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:22.423683 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c12549-8d8f-45c5-8229-934af3466041 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:22.424014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c12549-8d8f-45c5-8229-934af3466041 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:22.425185 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c12549-8d8f-45c5-8229-934af3466041 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:22.427198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c12549-8d8f-45c5-8229-934af3466041 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:22.429926 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f6a20-d88c-493a-a672-378ceaf43baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:22.430789 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f6a20-d88c-493a-a672-378ceaf43baf None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:22.431250 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f6a20-d88c-493a-a672-378ceaf43baf None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:22.431705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f6a20-d88c-493a-a672-378ceaf43baf None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:22.432710 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f6a20-d88c-493a-a672-378ceaf43baf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:22.433214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f6a20-d88c-493a-a672-378ceaf43baf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:22.434894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f6a20-d88c-493a-a672-378ceaf43baf None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:22.448737 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 412/832] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:25:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:22.449918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 421/833] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jun 6 10:25:22 2022] POST /identity/v3/endpoints => generated 327 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:25:22.450129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f6a20-d88c-493a-a672-378ceaf43baf None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:22.459899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c12549-8d8f-45c5-8229-934af3466041 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:22.465731 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:22.569758 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:22.608526 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c12549-8d8f-45c5-8229-934af3466041 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:22.654583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18457fa3-5726-456d-bccc-4602ea8d8557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:22.656010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18457fa3-5726-456d-bccc-4602ea8d8557 None admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:22.656295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18457fa3-5726-456d-bccc-4602ea8d8557 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:22.656571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18457fa3-5726-456d-bccc-4602ea8d8557 None admin] PATH_INFO: `/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/e9e358afe0974c0c8bd53104ebbff445/roles/95eccd30e39b4105ac6e8c0d9f973951` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:22.675676 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18457fa3-5726-456d-bccc-4602ea8d8557 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f082d7ed3694fd9998623f3d4a5fea9, user_id=e9e358afe0974c0c8bd53104ebbff445, role_id=95eccd30e39b4105ac6e8c0d9f973951)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:22.678203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18457fa3-5726-456d-bccc-4602ea8d8557 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:22.698177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 422/834] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:25:22 2022] PUT /identity/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/e9e358afe0974c0c8bd53104ebbff445/roles/95eccd30e39b4105ac6e8c0d9f973951 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:25:22.701475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f6a20-d88c-493a-a672-378ceaf43baf None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:22.754385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01ee75-fdf9-4895-b9a8-fcf42430a3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:22.755228 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01ee75-fdf9-4895-b9a8-fcf42430a3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:22.755507 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01ee75-fdf9-4895-b9a8-fcf42430a3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:22.755774 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01ee75-fdf9-4895-b9a8-fcf42430a3f8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:22.756538 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01ee75-fdf9-4895-b9a8-fcf42430a3f8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:22.757932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01ee75-fdf9-4895-b9a8-fcf42430a3f8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:22.772102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-5a01ee75-fdf9-4895-b9a8-fcf42430a3f8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 10:25:22.772102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:22.772102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:22.772102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:22.772102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:22.772102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:22.772102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:22.772102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:22.772102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 10:25:22.772102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:22.772102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:22.772102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:22.772102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:22.772102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:22.772102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:22.772102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:22.773374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:22.773374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:22.773374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:22.773374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:22.773374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:22.773374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:22.773374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:22.773374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:22.773374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:22.773374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:22.773374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:22.773374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:22.773374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:22.773374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:22.773374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:22.773374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:22.774235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:22.774235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:22.774235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:22.774235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:22.774235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:22.774235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:22.774235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:22.774235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:22.774235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:22.774235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:22.774235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:22.774235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:22.774235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:22.774235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 10:25:22.774235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:22.774235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 413/835] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Jun 6 10:25:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:22.780283 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18457fa3-5726-456d-bccc-4602ea8d8557 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:22.913880 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18457fa3-5726-456d-bccc-4602ea8d8557 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:22.947871 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:22.958475 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c549ca-d470-4d3d-bb72-c87c551c3d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:22.959169 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c549ca-d470-4d3d-bb72-c87c551c3d43 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:22.959459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c549ca-d470-4d3d-bb72-c87c551c3d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:22.959743 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c549ca-d470-4d3d-bb72-c87c551c3d43 None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:22.960968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c549ca-d470-4d3d-bb72-c87c551c3d43 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:22.961679 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c549ca-d470-4d3d-bb72-c87c551c3d43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:22.963724 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c549ca-d470-4d3d-bb72-c87c551c3d43 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:22.972652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 423/836] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jun 6 10:25:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:22.978355 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01ee75-fdf9-4895-b9a8-fcf42430a3f8 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:23.072740 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:23.095139 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01ee75-fdf9-4895-b9a8-fcf42430a3f8 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:23.134550 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e13609-c0f6-4b78-bb48-6ae0f547515c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:23.135235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e13609-c0f6-4b78-bb48-6ae0f547515c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:23.135516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e13609-c0f6-4b78-bb48-6ae0f547515c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:23.135812 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e13609-c0f6-4b78-bb48-6ae0f547515c None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:23.136557 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e13609-c0f6-4b78-bb48-6ae0f547515c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:23.136862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e13609-c0f6-4b78-bb48-6ae0f547515c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:23.138285 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e13609-c0f6-4b78-bb48-6ae0f547515c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:23.144900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 414/837] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jun 6 10:25:22 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:23.467948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7b883-a31c-419b-b89e-7436bd844503 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:23.468460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7b883-a31c-419b-b89e-7436bd844503 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:23.468538 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7b883-a31c-419b-b89e-7436bd844503 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:23.470131 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 424/838] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:23.476296 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1636a1-814d-4a8b-80a4-d32ff31e8eb8 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:23.476741 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1636a1-814d-4a8b-80a4-d32ff31e8eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:23.476916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1636a1-814d-4a8b-80a4-d32ff31e8eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:23.493796 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ce1636a1-814d-4a8b-80a4-d32ff31e8eb8 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:23.494988 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ce1636a1-814d-4a8b-80a4-d32ff31e8eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:23.566194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 415/839] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:23 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:23.589466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7b883-a31c-419b-b89e-7436bd844503 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:23.699627 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7b883-a31c-419b-b89e-7436bd844503 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:23.735220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aae205-9947-4eda-bbd7-d8b8629443f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:23.736043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aae205-9947-4eda-bbd7-d8b8629443f8 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:23.736900 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aae205-9947-4eda-bbd7-d8b8629443f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:23.737166 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aae205-9947-4eda-bbd7-d8b8629443f8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:23.738006 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aae205-9947-4eda-bbd7-d8b8629443f8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:23.739371 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aae205-9947-4eda-bbd7-d8b8629443f8 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:23.751211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-e9aae205-9947-4eda-bbd7-d8b8629443f8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 10:25:23.751211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:23.751211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:23.751211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:23.751211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:23.751211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:23.751211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:23.751211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:23.751211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 10:25:23.751211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:23.751211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:23.751211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:23.751211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:23.751211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:23.751211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:23.751211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:23.751211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:23.752285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:23.752285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:23.752285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:23.752285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:23.752285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:23.752285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:23.752285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:23.752285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:23.752285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:23.752285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:23.752285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:23.752285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:23.752285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:23.752285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:23.752285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:23.752285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:23.753115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:23.753115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:23.753115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:23.753115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:23.753115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:23.753115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:23.753115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:23.753115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:23.753115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:23.753115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:23.753115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:23.753115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:23.753115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 10:25:23.753115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:23.753115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 425/840] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:25:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:23.758808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1636a1-814d-4a8b-80a4-d32ff31e8eb8 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:23.820161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:23.859966 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1636a1-814d-4a8b-80a4-d32ff31e8eb8 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:23.895937 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1283b3-36f1-46ce-893a-0431bbfa521e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:23.896615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1283b3-36f1-46ce-893a-0431bbfa521e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:23.896899 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1283b3-36f1-46ce-893a-0431bbfa521e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:23.897175 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1283b3-36f1-46ce-893a-0431bbfa521e None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:23.897885 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1283b3-36f1-46ce-893a-0431bbfa521e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:23.898442 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1283b3-36f1-46ce-893a-0431bbfa521e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:23.899827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1283b3-36f1-46ce-893a-0431bbfa521e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:23.904883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 416/841] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:25:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:23.911125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aae205-9947-4eda-bbd7-d8b8629443f8 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:23.973870 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:24.021344 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aae205-9947-4eda-bbd7-d8b8629443f8 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:24.056749 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a51b3-3dce-4ee8-a113-820bd364f517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:24.057415 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a51b3-3dce-4ee8-a113-820bd364f517 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:24.057699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a51b3-3dce-4ee8-a113-820bd364f517 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:24.057982 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a51b3-3dce-4ee8-a113-820bd364f517 None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:24.058682 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a51b3-3dce-4ee8-a113-820bd364f517 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:24.058985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a51b3-3dce-4ee8-a113-820bd364f517 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:24.060376 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a51b3-3dce-4ee8-a113-820bd364f517 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:24.067337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 426/842] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:25:23 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:24.576361 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f6177-0771-48aa-acaf-0091650fc2ce None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:24.577324 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f6177-0771-48aa-acaf-0091650fc2ce None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:24.577810 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f6177-0771-48aa-acaf-0091650fc2ce None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:24.579641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 417/843] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:24.585340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd451996-24d7-4bde-9ee4-d6ba9a3899df None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:24.586183 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd451996-24d7-4bde-9ee4-d6ba9a3899df None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:24.586689 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd451996-24d7-4bde-9ee4-d6ba9a3899df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:24.605407 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-cd451996-24d7-4bde-9ee4-d6ba9a3899df None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:24.606911 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-cd451996-24d7-4bde-9ee4-d6ba9a3899df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:24.676841 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 427/844] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:24 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:24.701326 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f6177-0771-48aa-acaf-0091650fc2ce None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:24.818255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f6177-0771-48aa-acaf-0091650fc2ce None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:24.856326 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:24.861123 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14835ffd-4e37-474c-b0a8-fcf61591bda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:24.862639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14835ffd-4e37-474c-b0a8-fcf61591bda5 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:24.863208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14835ffd-4e37-474c-b0a8-fcf61591bda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:24.863640 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14835ffd-4e37-474c-b0a8-fcf61591bda5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:24.877161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14835ffd-4e37-474c-b0a8-fcf61591bda5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:25:24.886838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-14835ffd-4e37-474c-b0a8-fcf61591bda5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:25:24.886838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:24.886838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:24.886838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:24.886838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:24.886838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:24.886838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:24.886838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:24.886838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:24.886838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:24.886838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:24.886838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:24.886838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:25:24.886838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:25:24.886838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:25:24.886838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:25:24.887883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:25:24.887883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:25:24.887883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:25:24.887883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:25:24.887883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:24.887883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:24.887883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:24.887883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:24.887883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:24.887883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:24.887883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:24.887883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:24.887883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:24.887883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:24.887883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:24.887883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:24.888742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:24.888742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:24.888742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:24.888742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:24.888742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:25:24.888742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:25:24.888742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:25:24.888742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:25:24.888742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:25:24.888742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:25:24.888742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:25:24.888742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:24.888742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 418/845] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:25:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:24.895070 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd451996-24d7-4bde-9ee4-d6ba9a3899df None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:24.946594 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:24.998322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd451996-24d7-4bde-9ee4-d6ba9a3899df None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:25.040536 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1949918f-5d34-4bb9-8303-3b2f8230abae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:25.041218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949918f-5d34-4bb9-8303-3b2f8230abae None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:25.041514 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949918f-5d34-4bb9-8303-3b2f8230abae None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:25.041766 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949918f-5d34-4bb9-8303-3b2f8230abae None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:25.042484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1949918f-5d34-4bb9-8303-3b2f8230abae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:25.042788 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1949918f-5d34-4bb9-8303-3b2f8230abae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:25.044189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1949918f-5d34-4bb9-8303-3b2f8230abae None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:25.053505 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 428/846] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:25:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:25.058949 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14835ffd-4e37-474c-b0a8-fcf61591bda5 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:25.168180 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14835ffd-4e37-474c-b0a8-fcf61591bda5 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:25.211183 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc528acf-12d3-4d59-91ed-d667440f2233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:25.212141 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc528acf-12d3-4d59-91ed-d667440f2233 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:25.212420 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc528acf-12d3-4d59-91ed-d667440f2233 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:25.212690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc528acf-12d3-4d59-91ed-d667440f2233 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:25.214963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125eaf7-34e2-46ab-a6a0-d031ba709131 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:25.215249 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125eaf7-34e2-46ab-a6a0-d031ba709131 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:25.215531 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125eaf7-34e2-46ab-a6a0-d031ba709131 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:25.217323 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 429/847] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:25:25.223273 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0383e8-d215-4d11-b4b1-909086c079ea None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:25.223574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0383e8-d215-4d11-b4b1-909086c079ea None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:25.223838 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0383e8-d215-4d11-b4b1-909086c079ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:25.225631 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc528acf-12d3-4d59-91ed-d667440f2233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:25.227325 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc528acf-12d3-4d59-91ed-d667440f2233 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:25.241787 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-2a0383e8-d215-4d11-b4b1-909086c079ea None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:25.243106 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-2a0383e8-d215-4d11-b4b1-909086c079ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:25.249601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-cc528acf-12d3-4d59-91ed-d667440f2233 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:25.249601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:25.249601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:25.249601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:25.249601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:25.249601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:25.249601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:25.249601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:25.249601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:25.249601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:25.249601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:25.249601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:25.249601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:25.249601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:25.249601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:25.251580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:25.251580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:25.251580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:25.251580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:25.251580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:25.251580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:25.251580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:25.251580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:25.251580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:25.251580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:25.251580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:25.251580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:25.251580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:25.251580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:25.251580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:25.251580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:25.251580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:25.253244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:25.253244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:25.253244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:25.253244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:25.253244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 419/848] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:25.258051 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc528acf-12d3-4d59-91ed-d667440f2233 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:25.310270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:25.321807 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 430/849] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 10:25:25.346598 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0383e8-d215-4d11-b4b1-909086c079ea None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:25.376678 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc528acf-12d3-4d59-91ed-d667440f2233 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:25.404156 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:25.419775 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df4eaf-085f-4a39-ba01-33bbc8dfbe9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:25.420686 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df4eaf-085f-4a39-ba01-33bbc8dfbe9b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:25.420802 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df4eaf-085f-4a39-ba01-33bbc8dfbe9b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:25.421012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df4eaf-085f-4a39-ba01-33bbc8dfbe9b None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:25.421727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df4eaf-085f-4a39-ba01-33bbc8dfbe9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:25.422033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df4eaf-085f-4a39-ba01-33bbc8dfbe9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:25.423407 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df4eaf-085f-4a39-ba01-33bbc8dfbe9b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:25.440839 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 420/850] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:25.441746 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df4eaf-085f-4a39-ba01-33bbc8dfbe9b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:25.463695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0383e8-d215-4d11-b4b1-909086c079ea None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:25.502154 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c05b26-ae38-4040-84a9-edd8846ffd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:25.503494 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c05b26-ae38-4040-84a9-edd8846ffd99 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:25.504010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c05b26-ae38-4040-84a9-edd8846ffd99 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:25.504509 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c05b26-ae38-4040-84a9-edd8846ffd99 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:25.515799 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c05b26-ae38-4040-84a9-edd8846ffd99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:25:25.525372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-29c05b26-ae38-4040-84a9-edd8846ffd99 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:25:25.525372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:25.525372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:25.525372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:25.525372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:25.525372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:25.525372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:25.525372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:25.525372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:25.525372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:25.525372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:25.525372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:25.525372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:25:25.525372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:25:25.525372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:25:25.525372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:25:25.526396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:25:25.526396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:25:25.526396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:25:25.526396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:25:25.526396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:25.526396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:25.526396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:25.526396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:25.526396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:25.526396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:25.526396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:25.526396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:25.526396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:25.526396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:25.526396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:25.526396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:25.527216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:25.527216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:25.527216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:25.527216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:25.527216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:25:25.527216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:25:25.527216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:25:25.527216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:25:25.527216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:25:25.527216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:25:25.527216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:25:25.527216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:25.529356 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 431/851] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:25:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:25.535769 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c05b26-ae38-4040-84a9-edd8846ffd99 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:25.554425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:25.559105 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df4eaf-085f-4a39-ba01-33bbc8dfbe9b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:25.599632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de203221-3ee8-483d-8ea2-2d7fe3cd889f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:25.601280 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de203221-3ee8-483d-8ea2-2d7fe3cd889f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:25.601772 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de203221-3ee8-483d-8ea2-2d7fe3cd889f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:25.602242 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de203221-3ee8-483d-8ea2-2d7fe3cd889f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:25.609947 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de203221-3ee8-483d-8ea2-2d7fe3cd889f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:25.611629 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de203221-3ee8-483d-8ea2-2d7fe3cd889f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:25.626425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-de203221-3ee8-483d-8ea2-2d7fe3cd889f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 10:25:25.626425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:25.626425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:25.626425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:25.626425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:25.626425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:25.626425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:25.626425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:25.626425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:25.626425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:25.626425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:25.626425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:25.626425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:25:25.626425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:25:25.626425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:25:25.626425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:25:25.627451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:25.627451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:25.627451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:25.627451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:25.627451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:25.627451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:25.627451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:25.627451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:25.627451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:25.627451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:25.627451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:25.627451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:25.627451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:25.627451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:25.627451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:25.627451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:25.627451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:25.627451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:25.628535 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:25.628535 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:25.628535 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:25:25.628535 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:25:25.628535 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:25:25.628535 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:25:25.628535 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:25:25.628535 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:25:25.628535 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 10:25:25.628535 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:25.630487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 421/852] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:25:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:25.631866 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f45695-7aa8-4f68-8149-b5b1f7f87fbf None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:25.632143 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f45695-7aa8-4f68-8149-b5b1f7f87fbf None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:25.632438 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f45695-7aa8-4f68-8149-b5b1f7f87fbf None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:25.633858 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 422/853] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:25.638337 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f45695-7aa8-4f68-8149-b5b1f7f87fbf None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:25.662729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c05b26-ae38-4040-84a9-edd8846ffd99 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:25.664706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:25.699500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53542c-c87b-49e4-a7c7-35088ee5092a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:25.700305 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53542c-c87b-49e4-a7c7-35088ee5092a None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:25.700592 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53542c-c87b-49e4-a7c7-35088ee5092a None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:25.700877 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53542c-c87b-49e4-a7c7-35088ee5092a None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:25.701618 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53542c-c87b-49e4-a7c7-35088ee5092a None admin] RBAC: Adding query filter params (name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:25.701929 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53542c-c87b-49e4-a7c7-35088ee5092a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:25.703321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53542c-c87b-49e4-a7c7-35088ee5092a None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:25.712534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 432/854] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jun 6 10:25:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:25.713644 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17c7cf-9b66-4586-a008-87c0a1c8db32 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:25.713983 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17c7cf-9b66-4586-a008-87c0a1c8db32 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:25.714208 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17c7cf-9b66-4586-a008-87c0a1c8db32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:25.731345 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-4c17c7cf-9b66-4586-a008-87c0a1c8db32 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:25.732573 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-4c17c7cf-9b66-4586-a008-87c0a1c8db32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:25.764638 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f45695-7aa8-4f68-8149-b5b1f7f87fbf None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:25.799855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 433/855] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:25.806984 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17c7cf-9b66-4586-a008-87c0a1c8db32 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:25.809663 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a189db-7979-429c-aa3d-1559c8c5f5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:25.810576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a189db-7979-429c-aa3d-1559c8c5f5fe None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:25.811035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a189db-7979-429c-aa3d-1559c8c5f5fe None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:25.811494 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a189db-7979-429c-aa3d-1559c8c5f5fe None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:25.812610 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a189db-7979-429c-aa3d-1559c8c5f5fe None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:25.813114 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a189db-7979-429c-aa3d-1559c8c5f5fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:25.815852 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a189db-7979-429c-aa3d-1559c8c5f5fe None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:25.842396 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 423/856] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Jun 6 10:25:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:25.842396 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a189db-7979-429c-aa3d-1559c8c5f5fe None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:25.859395 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:25.915581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:25.925464 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17c7cf-9b66-4586-a008-87c0a1c8db32 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:25.948173 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a189db-7979-429c-aa3d-1559c8c5f5fe None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:25.962908 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c1ba6-7bb7-4e4a-b1b3-4d6d25da0c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:25.963935 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c1ba6-7bb7-4e4a-b1b3-4d6d25da0c05 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:25.964433 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c1ba6-7bb7-4e4a-b1b3-4d6d25da0c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:25.964649 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c1ba6-7bb7-4e4a-b1b3-4d6d25da0c05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:25.976714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c1ba6-7bb7-4e4a-b1b3-4d6d25da0c05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:25.978144 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c1ba6-7bb7-4e4a-b1b3-4d6d25da0c05 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:25.984212 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fc9c2-f2e4-4aac-96cc-fd146120d0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:25.985208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fc9c2-f2e4-4aac-96cc-fd146120d0b2 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:25.985208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fc9c2-f2e4-4aac-96cc-fd146120d0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:25.985423 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fc9c2-f2e4-4aac-96cc-fd146120d0b2 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:25.986135 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2fc9c2-f2e4-4aac-96cc-fd146120d0b2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:25.988162 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2fc9c2-f2e4-4aac-96cc-fd146120d0b2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:25.997706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-770c1ba6-7bb7-4e4a-b1b3-4d6d25da0c05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:25.997706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:25.997706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:25.997706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:25.997706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:25.997706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:25.997706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:25.997706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:25.997706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:25.997706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:25.997706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:25.997706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:25.997706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:25.997706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:25.997706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:25.998626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:25.998626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:25.998626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:25.998626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:25.998626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:25.998626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:25.998626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:25.998626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:25.998626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:25.998626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:25.998626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:25.998626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:25.998626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:25.998626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:25.998626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:25.998626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:25.998626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:25.999511 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:25.999511 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:25.999511 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:25.999511 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:26.001164 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 434/857] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:26.002773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c1ba6-7bb7-4e4a-b1b3-4d6d25da0c05 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:26.007963 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 424/858] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jun 6 10:25:25 2022] POST /identity/v3/services => generated 269 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:25:26.021469 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fc9c2-f2e4-4aac-96cc-fd146120d0b2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:26.133231 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c1ba6-7bb7-4e4a-b1b3-4d6d25da0c05 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:26.144572 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:26.183444 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029bae4-a8fe-4d72-8cad-5588261763e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:26.184309 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029bae4-a8fe-4d72-8cad-5588261763e4 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:26.184697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029bae4-a8fe-4d72-8cad-5588261763e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:26.185081 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029bae4-a8fe-4d72-8cad-5588261763e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:26.186103 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fc9c2-f2e4-4aac-96cc-fd146120d0b2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:26.199721 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:26.229486 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279fad75-25d1-4673-ac15-e48f17a704d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:26.230472 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fad75-25d1-4673-ac15-e48f17a704d3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:26.230968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fad75-25d1-4673-ac15-e48f17a704d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:26.231436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fad75-25d1-4673-ac15-e48f17a704d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:26.232499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279fad75-25d1-4673-ac15-e48f17a704d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:26.233000 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279fad75-25d1-4673-ac15-e48f17a704d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:26.234841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279fad75-25d1-4673-ac15-e48f17a704d3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:26.249522 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 425/859] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:26.255291 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279fad75-25d1-4673-ac15-e48f17a704d3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:26.271075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c029bae4-a8fe-4d72-8cad-5588261763e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:26.272921 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c029bae4-a8fe-4d72-8cad-5588261763e4 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:26.384667 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279fad75-25d1-4673-ac15-e48f17a704d3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:26.391167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 435/860] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:25:26 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 10:25:26.399744 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029bae4-a8fe-4d72-8cad-5588261763e4 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:26.427140 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:26.434890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c7ba6-18f7-42b3-9063-a635e44d0968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:26.436031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c7ba6-18f7-42b3-9063-a635e44d0968 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:26.436541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c7ba6-18f7-42b3-9063-a635e44d0968 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:26.437010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c7ba6-18f7-42b3-9063-a635e44d0968 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:26.444107 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:26.446814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c7ba6-18f7-42b3-9063-a635e44d0968 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:26.448691 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c7ba6-18f7-42b3-9063-a635e44d0968 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:26.466056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-8a6c7ba6-18f7-42b3-9063-a635e44d0968 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 10:25:26.466056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:26.466056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:26.466056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:26.466056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:26.466056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:26.466056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:26.466056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:26.466056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:26.466056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:26.466056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:26.466056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:26.466056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:25:26.466056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:25:26.466056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:25:26.466056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:25:26.467066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:26.467066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:26.467066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:26.467066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:26.467066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:26.467066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:26.467066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:26.467066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:26.467066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:26.467066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:26.467066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:26.467066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:26.467066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:26.467066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:26.467066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:26.467066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:26.467066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:26.467066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:26.468463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:26.468463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:26.468463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:25:26.468463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:25:26.468463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:25:26.468463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:25:26.468463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:25:26.468463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:25:26.468463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 10:25:26.468463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:26.468463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 426/861] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jun 6 10:25:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:26.469078 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c1b40-c87e-4d3b-953c-e423e046a309 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:26.469330 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c1b40-c87e-4d3b-953c-e423e046a309 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:26.469492 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c1b40-c87e-4d3b-953c-e423e046a309 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:26.471438 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 427/862] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:26.475547 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c1b40-c87e-4d3b-953c-e423e046a309 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:26.509474 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029bae4-a8fe-4d72-8cad-5588261763e4 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:26.544903 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b29fe-4270-4d81-8677-a21221009686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:26.545292 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b29fe-4270-4d81-8677-a21221009686 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:26.545574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b29fe-4270-4d81-8677-a21221009686 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:26.545847 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b29fe-4270-4d81-8677-a21221009686 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:26.558473 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b29fe-4270-4d81-8677-a21221009686 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:26.560121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b29fe-4270-4d81-8677-a21221009686 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:26.578701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-2f4b29fe-4270-4d81-8677-a21221009686 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:26.578701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:26.578701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:26.578701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:26.578701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:26.578701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:26.578701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:26.578701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:26.578701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:26.578701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:26.578701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:26.578701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:26.578701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:26.578701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:26.578701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:26.579624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:26.579624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:26.579624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:26.579624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:26.579624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:26.579624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:26.579624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:26.579624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:26.579624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:26.579624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:26.579624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:26.579624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:26.579624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:26.579624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:26.579624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:26.579624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:26.579624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:26.581152 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:26.581152 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:26.581152 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:26.581152 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:26.581152 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 436/863] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:26.582162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb1839-e98a-4cae-ad55-5580f2ae7676 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:26.582444 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb1839-e98a-4cae-ad55-5580f2ae7676 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:26.582721 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb1839-e98a-4cae-ad55-5580f2ae7676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:26.586845 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c1b40-c87e-4d3b-953c-e423e046a309 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:26.602797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-13fb1839-e98a-4cae-ad55-5580f2ae7676 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:26.604184 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-13fb1839-e98a-4cae-ad55-5580f2ae7676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:26.629740 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8978a9-94d2-406e-a583-1065b07309a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:26.630541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8978a9-94d2-406e-a583-1065b07309a2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:26.630820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8978a9-94d2-406e-a583-1065b07309a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:26.631187 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8978a9-94d2-406e-a583-1065b07309a2 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:26.632058 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8978a9-94d2-406e-a583-1065b07309a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:26.632365 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8978a9-94d2-406e-a583-1065b07309a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:26.633902 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8978a9-94d2-406e-a583-1065b07309a2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:26.653610 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 428/864] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Jun 6 10:25:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:26.654501 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8978a9-94d2-406e-a583-1065b07309a2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:26.678704 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 437/865] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:26 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:26.680612 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb1839-e98a-4cae-ad55-5580f2ae7676 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:26.706971 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:26.719268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:26.761186 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8978a9-94d2-406e-a583-1065b07309a2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:26.804063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92dce0a-e4dd-447b-97dc-935999f4b8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:26.804516 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb1839-e98a-4cae-ad55-5580f2ae7676 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:26.804724 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92dce0a-e4dd-447b-97dc-935999f4b8ff None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:26.805003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92dce0a-e4dd-447b-97dc-935999f4b8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:26.805275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92dce0a-e4dd-447b-97dc-935999f4b8ff None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:26.805989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92dce0a-e4dd-447b-97dc-935999f4b8ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:26.806293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92dce0a-e4dd-447b-97dc-935999f4b8ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:26.807656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92dce0a-e4dd-447b-97dc-935999f4b8ff None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:26.819428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 429/866] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:26.820265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92dce0a-e4dd-447b-97dc-935999f4b8ff None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:26.847381 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d7365-61c5-468a-9c2d-2f4ce6a17191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:26.848059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d7365-61c5-468a-9c2d-2f4ce6a17191 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:26.848344 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d7365-61c5-468a-9c2d-2f4ce6a17191 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:26.848616 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d7365-61c5-468a-9c2d-2f4ce6a17191 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:26.923472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d7365-61c5-468a-9c2d-2f4ce6a17191 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:26.924963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d7365-61c5-468a-9c2d-2f4ce6a17191 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:26.935617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92dce0a-e4dd-447b-97dc-935999f4b8ff None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:26.946100 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:26.953471 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:26.979342 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b9014-49c7-4965-aaec-a4c12b54f47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:26.980416 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b9014-49c7-4965-aaec-a4c12b54f47b None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:26.981116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b9014-49c7-4965-aaec-a4c12b54f47b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:26.981555 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b9014-49c7-4965-aaec-a4c12b54f47b None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:26.982422 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b9014-49c7-4965-aaec-a4c12b54f47b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:26.984393 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b9014-49c7-4965-aaec-a4c12b54f47b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:27.004498 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 430/867] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jun 6 10:25:26 2022] POST /identity/v3/services => generated 251 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:25:27.005624 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b9014-49c7-4965-aaec-a4c12b54f47b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:27.042474 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 438/868] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:25:26 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 10:25:27.047758 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d7365-61c5-468a-9c2d-2f4ce6a17191 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:27.165356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b9014-49c7-4965-aaec-a4c12b54f47b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:27.165723 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d7365-61c5-468a-9c2d-2f4ce6a17191 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:27.198690 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:27.213390 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706bdec-7c6d-4b11-beb6-1dbd7d3d831c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:27.214516 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706bdec-7c6d-4b11-beb6-1dbd7d3d831c None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:27.214970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706bdec-7c6d-4b11-beb6-1dbd7d3d831c None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:27.215387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706bdec-7c6d-4b11-beb6-1dbd7d3d831c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:27.217977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc31092-f963-477b-a1d7-793723e2ca3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:27.218835 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc31092-f963-477b-a1d7-793723e2ca3d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:27.219622 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc31092-f963-477b-a1d7-793723e2ca3d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:27.219985 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc31092-f963-477b-a1d7-793723e2ca3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:27.229702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706bdec-7c6d-4b11-beb6-1dbd7d3d831c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:27.229936 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc31092-f963-477b-a1d7-793723e2ca3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:27.231423 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc31092-f963-477b-a1d7-793723e2ca3d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:27.231891 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706bdec-7c6d-4b11-beb6-1dbd7d3d831c None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:27.249369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-6cc31092-f963-477b-a1d7-793723e2ca3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:27.249369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:27.249369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:27.249369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:27.249369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:27.249369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:27.249369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:27.249369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:27.249369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:27.249369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:27.249369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:27.249369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:27.249369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:27.249369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:27.249369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:27.250359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:27.250359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:27.250359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:27.250359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:27.250359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:27.250359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:27.250359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:27.250359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:27.250359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:27.250359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:27.250359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:27.250359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:27.250359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:27.250359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:27.250359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:27.250359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:27.250359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:27.251776 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:27.251776 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:27.251776 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:27.251776 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:27.251776 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:27.251776 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:27.251776 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:27.251776 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:27.251776 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 431/869] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:27.257439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-b706bdec-7c6d-4b11-beb6-1dbd7d3d831c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:27.257439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:27.257439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:27.257439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:27.257439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:27.257439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:27.257439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:27.257439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:27.257439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:27.257439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:27.257439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:27.257439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:27.257439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:27.257439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:27.257439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:27.258733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:27.258733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:27.258733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:27.258733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:27.258733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:27.258733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:27.258733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:27.258733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:27.258733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:27.258733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:27.258733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:27.258733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:27.258733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:27.258733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:27.258733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:27.258733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:27.258733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:27.262229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:27.262229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:27.262229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:27.262229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:27.263696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc31092-f963-477b-a1d7-793723e2ca3d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:27.263894 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 439/870] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:27.263894 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706bdec-7c6d-4b11-beb6-1dbd7d3d831c None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:27.292190 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:27.389050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc31092-f963-477b-a1d7-793723e2ca3d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:27.407178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706bdec-7c6d-4b11-beb6-1dbd7d3d831c None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:27.427567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aba6e8-2c72-4cf2-8496-046e81cf8cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:27.428093 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aba6e8-2c72-4cf2-8496-046e81cf8cba None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:27.428373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aba6e8-2c72-4cf2-8496-046e81cf8cba None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:27.428650 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aba6e8-2c72-4cf2-8496-046e81cf8cba None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:27.429431 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aba6e8-2c72-4cf2-8496-046e81cf8cba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:27.429801 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aba6e8-2c72-4cf2-8496-046e81cf8cba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:27.431231 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aba6e8-2c72-4cf2-8496-046e81cf8cba None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:27.443949 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 432/871] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:25:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:27.451372 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aba6e8-2c72-4cf2-8496-046e81cf8cba None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:27.453095 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8219c13-f398-493c-bd20-a034a29a86cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:27.453999 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8219c13-f398-493c-bd20-a034a29a86cb None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:27.454489 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8219c13-f398-493c-bd20-a034a29a86cb None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:27.455869 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8219c13-f398-493c-bd20-a034a29a86cb None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:27.458030 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8219c13-f398-493c-bd20-a034a29a86cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:27.458555 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8219c13-f398-493c-bd20-a034a29a86cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:27.460158 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8219c13-f398-493c-bd20-a034a29a86cb None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:27.477186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 440/872] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:27.483833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8219c13-f398-493c-bd20-a034a29a86cb None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:27.524425 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:27.565867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:27.584163 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aba6e8-2c72-4cf2-8496-046e81cf8cba None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:27.613870 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8219c13-f398-493c-bd20-a034a29a86cb None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:27.620212 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c9dc98-9f29-499c-b827-7b468b1906c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:27.620875 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9dc98-9f29-499c-b827-7b468b1906c5 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:27.621157 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9dc98-9f29-499c-b827-7b468b1906c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:27.621427 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9dc98-9f29-499c-b827-7b468b1906c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:27.622259 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c9dc98-9f29-499c-b827-7b468b1906c5 None admin] RBAC: Adding query filter params (role.id=0f929204eb7b4ea8859338ed2d3ff75b, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=fede9e52ad274285b09d748a684ede38) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:27.622629 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c9dc98-9f29-499c-b827-7b468b1906c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:27.624092 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c9dc98-9f29-499c-b827-7b468b1906c5 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:27.640390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 433/873] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:25:27 2022] GET /identity/v3/role_assignments?user.id=fede9e52ad274285b09d748a684ede38&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=0f929204eb7b4ea8859338ed2d3ff75b => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:27.664212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb60156-de25-4a64-8ca1-a5c6c20d117a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:27.665018 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb60156-de25-4a64-8ca1-a5c6c20d117a None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:27.665376 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb60156-de25-4a64-8ca1-a5c6c20d117a None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:27.665708 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb60156-de25-4a64-8ca1-a5c6c20d117a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:27.674854 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb60156-de25-4a64-8ca1-a5c6c20d117a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:27.676445 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb60156-de25-4a64-8ca1-a5c6c20d117a None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:27.692662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-7eb60156-de25-4a64-8ca1-a5c6c20d117a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:27.692662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:27.692662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:27.692662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:27.692662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:27.692662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:27.692662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:27.692662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:27.692662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:27.692662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:27.692662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:27.692662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:27.692662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:27.692662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:27.692662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:27.694041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:27.694041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:27.694041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:27.694041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:27.694041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:27.694041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:27.694041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:27.694041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:27.694041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:27.694041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:27.694041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:27.694041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:27.694041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:27.694041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:27.694041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:27.694041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:27.694041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:27.694931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:27.694931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:27.694931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:27.694931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:27.694931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:27.694931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:27.694931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:27.694931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:27.694931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 441/874] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:27.700027 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c9dc98-9f29-499c-b827-7b468b1906c5 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:27.817829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c9dc98-9f29-499c-b827-7b468b1906c5 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:27.856762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd5a85-2b45-4854-977f-7eb69907af8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:27.857779 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd5a85-2b45-4854-977f-7eb69907af8c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:27.858252 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd5a85-2b45-4854-977f-7eb69907af8c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:27.858718 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd5a85-2b45-4854-977f-7eb69907af8c None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:27.859700 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd5a85-2b45-4854-977f-7eb69907af8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:27.860307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd5a85-2b45-4854-977f-7eb69907af8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:27.862042 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd5a85-2b45-4854-977f-7eb69907af8c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:27.877745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 434/875] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:25:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:27.884035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb60156-de25-4a64-8ca1-a5c6c20d117a None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:27.987616 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:27.994900 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb60156-de25-4a64-8ca1-a5c6c20d117a None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:28.030534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3e3de-7d6e-48d4-b33f-a6903a6942dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:28.031212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3e3de-7d6e-48d4-b33f-a6903a6942dd None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:28.031493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3e3de-7d6e-48d4-b33f-a6903a6942dd None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:28.031767 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3e3de-7d6e-48d4-b33f-a6903a6942dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:28.032657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3e3de-7d6e-48d4-b33f-a6903a6942dd None admin] RBAC: Adding query filter params (role.id=95eccd30e39b4105ac6e8c0d9f973951, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=e9e358afe0974c0c8bd53104ebbff445) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:28.033499 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3e3de-7d6e-48d4-b33f-a6903a6942dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:28.035071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3e3de-7d6e-48d4-b33f-a6903a6942dd None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:28.042516 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 442/876] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:25:27 2022] GET /identity/v3/role_assignments?user.id=e9e358afe0974c0c8bd53104ebbff445&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=95eccd30e39b4105ac6e8c0d9f973951 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:28.375882 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ed3b3-e16a-4926-aeb7-b3db56e2aa47 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:28.376056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ed3b3-e16a-4926-aeb7-b3db56e2aa47 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:28.376328 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ed3b3-e16a-4926-aeb7-b3db56e2aa47 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:28.378430 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 435/877] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:28.385977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bca92-c297-4d03-8f8a-1104c9823504 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:28.386500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bca92-c297-4d03-8f8a-1104c9823504 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:28.386980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bca92-c297-4d03-8f8a-1104c9823504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:28.404926 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-1a3bca92-c297-4d03-8f8a-1104c9823504 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:28.406420 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-1a3bca92-c297-4d03-8f8a-1104c9823504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:28.477499 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 443/878] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:28 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:28.499861 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ed3b3-e16a-4926-aeb7-b3db56e2aa47 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:28.513605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:28.609937 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ed3b3-e16a-4926-aeb7-b3db56e2aa47 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:28.646747 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79929c9-0037-4de6-938b-6bdf4942c5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:28.647470 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79929c9-0037-4de6-938b-6bdf4942c5ac None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:28.647755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79929c9-0037-4de6-938b-6bdf4942c5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:28.648300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79929c9-0037-4de6-938b-6bdf4942c5ac None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:28.648956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79929c9-0037-4de6-938b-6bdf4942c5ac None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:28.650328 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79929c9-0037-4de6-938b-6bdf4942c5ac None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:28.663910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-e79929c9-0037-4de6-938b-6bdf4942c5ac None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 10:25:28.663910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:28.663910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:28.663910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:28.663910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:28.663910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:28.663910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:28.663910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:28.663910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 10:25:28.663910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:28.663910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:28.663910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:28.663910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:28.663910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:28.663910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:28.663910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:28.664916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:28.664916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:28.664916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:28.664916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:28.664916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:28.664916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:28.664916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:28.664916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:28.664916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:28.664916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:28.664916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:28.664916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:28.664916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:28.664916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:28.664916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:28.664916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:28.665762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:28.665762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:28.665762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:28.665762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:28.665762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:28.665762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:28.665762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:28.665762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:28.665762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:28.665762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:28.665762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:28.665762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:28.665762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:28.665762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 10:25:28.665762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:28.665762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 436/879] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Jun 6 10:25:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:28.673989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bca92-c297-4d03-8f8a-1104c9823504 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:28.766483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:28.779309 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bca92-c297-4d03-8f8a-1104c9823504 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:28.815600 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25313cf5-1760-42bb-be8a-6902658a36d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:28.816658 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25313cf5-1760-42bb-be8a-6902658a36d7 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:28.817137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25313cf5-1760-42bb-be8a-6902658a36d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:28.817606 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25313cf5-1760-42bb-be8a-6902658a36d7 None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:28.818513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25313cf5-1760-42bb-be8a-6902658a36d7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:28.819019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25313cf5-1760-42bb-be8a-6902658a36d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:28.820690 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25313cf5-1760-42bb-be8a-6902658a36d7 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:28.826196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 444/880] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jun 6 10:25:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:28.832628 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79929c9-0037-4de6-938b-6bdf4942c5ac None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:28.924351 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:28.951255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79929c9-0037-4de6-938b-6bdf4942c5ac None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:28.992053 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9125c78-78be-4702-9f50-66f43c7d7113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:28.993115 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9125c78-78be-4702-9f50-66f43c7d7113 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:28.993613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9125c78-78be-4702-9f50-66f43c7d7113 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:28.994098 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9125c78-78be-4702-9f50-66f43c7d7113 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:28.995040 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9125c78-78be-4702-9f50-66f43c7d7113 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:28.995554 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9125c78-78be-4702-9f50-66f43c7d7113 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:28.997161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9125c78-78be-4702-9f50-66f43c7d7113 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:29.003990 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 437/881] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jun 6 10:25:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:29.015816 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25313cf5-1760-42bb-be8a-6902658a36d7 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:29.116752 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25313cf5-1760-42bb-be8a-6902658a36d7 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:29.152428 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d5ab0-b6ea-4c44-81dd-e6e8daf71781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:29.153098 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d5ab0-b6ea-4c44-81dd-e6e8daf71781 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:29.153386 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d5ab0-b6ea-4c44-81dd-e6e8daf71781 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:29.153668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d5ab0-b6ea-4c44-81dd-e6e8daf71781 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:29.154454 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d5ab0-b6ea-4c44-81dd-e6e8daf71781 None admin] RBAC: Adding query filter params (interface=public, service_id=16408d81689c473fbc4746143dd12d82, region_id=RegionOne) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:29.154771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d5ab0-b6ea-4c44-81dd-e6e8daf71781 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:29.156165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d5ab0-b6ea-4c44-81dd-e6e8daf71781 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:29.164681 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 445/882] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jun 6 10:25:29 2022] GET /identity/v3/endpoints?service_id=16408d81689c473fbc4746143dd12d82&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:29.169730 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9125c78-78be-4702-9f50-66f43c7d7113 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:29.288895 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9125c78-78be-4702-9f50-66f43c7d7113 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:29.324322 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2145c6-9101-4aa9-80cd-e5e6ef23fe0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:29.325437 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2145c6-9101-4aa9-80cd-e5e6ef23fe0e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:29.325936 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2145c6-9101-4aa9-80cd-e5e6ef23fe0e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:29.326430 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2145c6-9101-4aa9-80cd-e5e6ef23fe0e None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:29.327328 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2145c6-9101-4aa9-80cd-e5e6ef23fe0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:29.329038 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2145c6-9101-4aa9-80cd-e5e6ef23fe0e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:29.336274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 438/883] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jun 6 10:25:29 2022] GET /identity/v3/services => generated 1268 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:25:29.436076 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7095b19-7c20-4adb-8aaa-05543b000e6e None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:29.436713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7095b19-7c20-4adb-8aaa-05543b000e6e None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:29.437204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7095b19-7c20-4adb-8aaa-05543b000e6e None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:29.439131 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 446/884] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:29.445342 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c9ebe-a92d-4cb2-9170-5bf767ab972d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:29.445631 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c9ebe-a92d-4cb2-9170-5bf767ab972d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:29.445912 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c9ebe-a92d-4cb2-9170-5bf767ab972d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:29.451297 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:29.466164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-170c9ebe-a92d-4cb2-9170-5bf767ab972d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:29.467380 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-170c9ebe-a92d-4cb2-9170-5bf767ab972d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:29.551980 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 439/885] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 10:25:29.569961 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7095b19-7c20-4adb-8aaa-05543b000e6e None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:29.595715 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:29.687520 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7095b19-7c20-4adb-8aaa-05543b000e6e None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:29.724822 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7e5d9-b49c-4dc4-bdfd-746e51b2f39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:29.725676 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7e5d9-b49c-4dc4-bdfd-746e51b2f39d None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:29.725954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7e5d9-b49c-4dc4-bdfd-746e51b2f39d None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:29.726226 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7e5d9-b49c-4dc4-bdfd-746e51b2f39d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:29.726915 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7e5d9-b49c-4dc4-bdfd-746e51b2f39d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:29.728316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7e5d9-b49c-4dc4-bdfd-746e51b2f39d None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:29.741412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-dbb7e5d9-b49c-4dc4-bdfd-746e51b2f39d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 10:25:29.741412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:29.741412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:29.741412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:29.741412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:29.741412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:29.741412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:29.741412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:29.741412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 10:25:29.741412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:29.741412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:29.741412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:29.741412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:29.741412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:29.741412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:29.741412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:29.741412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:29.742466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:29.742466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:29.742466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:29.742466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:29.742466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:29.742466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:29.742466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:29.742466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:29.742466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:29.742466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:29.742466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:29.742466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:29.742466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:29.742466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:29.742466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:29.742466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:29.743308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:29.743308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:29.743308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:29.743308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:29.743308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:29.743308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:29.743308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:29.743308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:29.743308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:29.743308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:29.743308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:29.743308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:29.743308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 10:25:29.743308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:29.743308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 447/886] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:25:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:29.750601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c9ebe-a92d-4cb2-9170-5bf767ab972d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:29.851952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c9ebe-a92d-4cb2-9170-5bf767ab972d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:29.891294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:29.897149 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48ea52-f098-4645-9af5-26db1b710b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:29.897845 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48ea52-f098-4645-9af5-26db1b710b4d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:29.898402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48ea52-f098-4645-9af5-26db1b710b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:29.898678 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48ea52-f098-4645-9af5-26db1b710b4d None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:29.899419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48ea52-f098-4645-9af5-26db1b710b4d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:29.899726 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48ea52-f098-4645-9af5-26db1b710b4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:29.901203 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48ea52-f098-4645-9af5-26db1b710b4d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:29.907769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 440/887] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:25:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:29.914291 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7e5d9-b49c-4dc4-bdfd-746e51b2f39d None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:30.003344 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:30.028070 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7e5d9-b49c-4dc4-bdfd-746e51b2f39d None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:30.064399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f60a26-d5df-4023-82b7-0e34efb1794a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:30.065400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f60a26-d5df-4023-82b7-0e34efb1794a None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:30.065685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f60a26-d5df-4023-82b7-0e34efb1794a None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:30.065961 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f60a26-d5df-4023-82b7-0e34efb1794a None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:30.066665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f60a26-d5df-4023-82b7-0e34efb1794a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:30.066967 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f60a26-d5df-4023-82b7-0e34efb1794a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:30.068364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f60a26-d5df-4023-82b7-0e34efb1794a None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:30.073865 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 448/888] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:25:29 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:30.079601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48ea52-f098-4645-9af5-26db1b710b4d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:30.192075 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48ea52-f098-4645-9af5-26db1b710b4d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:30.228771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746c9a0-b565-445b-8181-44dbb6cb1c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:30.229430 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746c9a0-b565-445b-8181-44dbb6cb1c24 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:30.229711 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746c9a0-b565-445b-8181-44dbb6cb1c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:30.229990 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746c9a0-b565-445b-8181-44dbb6cb1c24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:30.230779 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0746c9a0-b565-445b-8181-44dbb6cb1c24 None admin] RBAC: Adding query filter params (interface=public, service_id=3b5337dcc532403ea1c1933b793dc0a6, region_id=RegionOne) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:30.231089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0746c9a0-b565-445b-8181-44dbb6cb1c24 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:30.232487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0746c9a0-b565-445b-8181-44dbb6cb1c24 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:30.238507 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 441/889] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jun 6 10:25:30 2022] GET /identity/v3/endpoints?service_id=3b5337dcc532403ea1c1933b793dc0a6&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:30.243851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f60a26-d5df-4023-82b7-0e34efb1794a None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:30.278001 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97fe6a-21b2-4895-828f-68212405c156 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:30.278590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97fe6a-21b2-4895-828f-68212405c156 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:30.279085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97fe6a-21b2-4895-828f-68212405c156 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:30.280934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 442/890] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:30.286994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d406f70-2bd8-4809-a095-9e9a5cee33b2 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:30.287561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d406f70-2bd8-4809-a095-9e9a5cee33b2 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:30.288028 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d406f70-2bd8-4809-a095-9e9a5cee33b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:30.305262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-5d406f70-2bd8-4809-a095-9e9a5cee33b2 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:30.306705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5d406f70-2bd8-4809-a095-9e9a5cee33b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:30.353264 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f60a26-d5df-4023-82b7-0e34efb1794a None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:30.392316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:30.400010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea163a-de5c-444b-8caa-f073aa610cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:30.400890 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea163a-de5c-444b-8caa-f073aa610cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:30.401317 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea163a-de5c-444b-8caa-f073aa610cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:30.401744 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea163a-de5c-444b-8caa-f073aa610cf4 None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:30.402558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea163a-de5c-444b-8caa-f073aa610cf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:30.404114 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea163a-de5c-444b-8caa-f073aa610cf4 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:30.404567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 443/891] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 10:25:30.412403 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 449/892] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jun 6 10:25:30 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:25:30.436425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d406f70-2bd8-4809-a095-9e9a5cee33b2 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:30.511840 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca861faa-ad39-4601-a9f9-80e4304c055e None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:30.512451 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca861faa-ad39-4601-a9f9-80e4304c055e None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:30.512950 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca861faa-ad39-4601-a9f9-80e4304c055e None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:30.514987 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 450/893] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:25:30.520853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b8b26-85a6-4fbe-ac00-76107cb9bf48 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:30.521209 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b8b26-85a6-4fbe-ac00-76107cb9bf48 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:30.521481 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b8b26-85a6-4fbe-ac00-76107cb9bf48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:30.539736 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-e26b8b26-85a6-4fbe-ac00-76107cb9bf48 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:30.541255 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-e26b8b26-85a6-4fbe-ac00-76107cb9bf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:30.544547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:30.561765 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d406f70-2bd8-4809-a095-9e9a5cee33b2 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:30.600988 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8816a7-0cb0-4c4c-8421-7a8882b33e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:30.601801 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8816a7-0cb0-4c4c-8421-7a8882b33e18 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:30.602082 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8816a7-0cb0-4c4c-8421-7a8882b33e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:30.602359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8816a7-0cb0-4c4c-8421-7a8882b33e18 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:30.603020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8816a7-0cb0-4c4c-8421-7a8882b33e18 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:30.604423 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8816a7-0cb0-4c4c-8421-7a8882b33e18 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:30.617759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-bc8816a7-0cb0-4c4c-8421-7a8882b33e18 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 10:25:30.617759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:30.617759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:30.617759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:30.617759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:30.617759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:30.617759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:30.617759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:30.617759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 10:25:30.617759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:30.617759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:30.617759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:30.617759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:30.617759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:30.617759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:30.617759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:30.617759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:30.618792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:30.618792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:30.618792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:30.618792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:30.618792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:30.618792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:30.618792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:30.618792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:30.618792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:30.618792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:30.618792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:30.618792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:30.618792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:30.618792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:30.618792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:30.618792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:30.619634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:30.619634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:30.619634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:30.619634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:30.619634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:30.619634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:30.619634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:30.619634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:30.619634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:30.619634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:30.619634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:30.619634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:30.619634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 10:25:30.619634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:30.619634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 444/894] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jun 6 10:25:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:30.625172 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8816a7-0cb0-4c4c-8421-7a8882b33e18 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:30.631612 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 451/895] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 10:25:30.656319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b8b26-85a6-4fbe-ac00-76107cb9bf48 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:30.689880 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:30.739179 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8816a7-0cb0-4c4c-8421-7a8882b33e18 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:30.775259 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb29d5e-8117-4e5c-9ea0-4498061fedc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:30.776038 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb29d5e-8117-4e5c-9ea0-4498061fedc3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:30.776135 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb29d5e-8117-4e5c-9ea0-4498061fedc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:30.776417 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb29d5e-8117-4e5c-9ea0-4498061fedc3 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:30.777130 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb29d5e-8117-4e5c-9ea0-4498061fedc3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:30.777438 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb29d5e-8117-4e5c-9ea0-4498061fedc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:30.778827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb29d5e-8117-4e5c-9ea0-4498061fedc3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:30.784843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 445/896] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jun 6 10:25:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:30.790238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb29d5e-8117-4e5c-9ea0-4498061fedc3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:30.804999 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b8b26-85a6-4fbe-ac00-76107cb9bf48 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:30.843735 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4c41a-b930-484c-a7a7-071c5aef15d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:30.845088 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4c41a-b930-484c-a7a7-071c5aef15d9 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:30.846302 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4c41a-b930-484c-a7a7-071c5aef15d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:30.846831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4c41a-b930-484c-a7a7-071c5aef15d9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:30.855407 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:30.859388 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4c41a-b930-484c-a7a7-071c5aef15d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:25:30.869981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-3fb4c41a-b930-484c-a7a7-071c5aef15d9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 10:25:30.869981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:30.869981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:30.869981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:30.869981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:30.869981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:30.869981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:30.869981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:30.869981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:30.869981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:30.869981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:30.869981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:30.869981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:25:30.869981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:25:30.869981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:25:30.869981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:25:30.870960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:25:30.870960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:25:30.870960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:25:30.870960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:25:30.870960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:30.870960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:30.870960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:30.870960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:30.870960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:30.870960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:30.870960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:30.870960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:30.870960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:30.870960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:30.870960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:30.870960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:30.871861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:30.871861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:30.871861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:30.871861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:30.871861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:25:30.871861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:25:30.871861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:25:30.871861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:25:30.871861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:25:30.871861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:25:30.871861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 10:25:30.871861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:30.880158 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 452/897] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Jun 6 10:25:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:30.881585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4c41a-b930-484c-a7a7-071c5aef15d9 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:30.895537 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb29d5e-8117-4e5c-9ea0-4498061fedc3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:30.913826 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:30.932806 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb32f64-36d7-4d91-aef9-b22191c8f765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:30.933000 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb32f64-36d7-4d91-aef9-b22191c8f765 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:30.933266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb32f64-36d7-4d91-aef9-b22191c8f765 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:30.933549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb32f64-36d7-4d91-aef9-b22191c8f765 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:30.934323 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb32f64-36d7-4d91-aef9-b22191c8f765 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:30.934636 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb32f64-36d7-4d91-aef9-b22191c8f765 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:30.936050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb32f64-36d7-4d91-aef9-b22191c8f765 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:30.947061 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 446/898] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jun 6 10:25:30 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:31.003261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4c41a-b930-484c-a7a7-071c5aef15d9 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:31.043093 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a8bcc-d131-4646-ab34-a8376d5aa311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:31.044153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a8bcc-d131-4646-ab34-a8376d5aa311 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:31.044688 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a8bcc-d131-4646-ab34-a8376d5aa311 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:31.045180 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a8bcc-d131-4646-ab34-a8376d5aa311 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:31.046136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a8bcc-d131-4646-ab34-a8376d5aa311 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:31.046642 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a8bcc-d131-4646-ab34-a8376d5aa311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:31.048284 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a8bcc-d131-4646-ab34-a8376d5aa311 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:31.063077 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 453/899] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Jun 6 10:25:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:31.065744 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb32f64-36d7-4d91-aef9-b22191c8f765 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:31.177554 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb32f64-36d7-4d91-aef9-b22191c8f765 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:31.213711 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350115a-fa1c-4223-a871-5b34f2afcd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:31.214517 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350115a-fa1c-4223-a871-5b34f2afcd29 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:31.214801 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350115a-fa1c-4223-a871-5b34f2afcd29 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:31.215077 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350115a-fa1c-4223-a871-5b34f2afcd29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:31.226101 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350115a-fa1c-4223-a871-5b34f2afcd29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:31.227577 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350115a-fa1c-4223-a871-5b34f2afcd29 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:31.245656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-9350115a-fa1c-4223-a871-5b34f2afcd29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:31.245656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:31.245656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:31.245656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:31.245656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:31.245656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:31.245656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:31.245656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:31.245656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:31.245656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:31.245656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:31.245656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:31.245656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:31.245656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:31.245656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:31.246666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:31.246666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:31.246666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:31.246666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:31.246666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:31.246666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:31.246666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:31.246666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:31.246666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:31.246666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:31.246666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:31.246666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:31.246666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:31.246666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:31.246666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:31.246666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:31.246666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:31.247920 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:31.247920 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:31.247920 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:31.247920 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:31.247920 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 447/900] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:31.254529 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a8bcc-d131-4646-ab34-a8376d5aa311 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:31.321035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:31.363703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a8bcc-d131-4646-ab34-a8376d5aa311 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:31.407450 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6486d6-1642-4fd3-830a-1ca41693f2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:31.408549 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6486d6-1642-4fd3-830a-1ca41693f2ba None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:31.408930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6486d6-1642-4fd3-830a-1ca41693f2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:31.409306 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6486d6-1642-4fd3-830a-1ca41693f2ba None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:31.410139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6486d6-1642-4fd3-830a-1ca41693f2ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:31.410541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6486d6-1642-4fd3-830a-1ca41693f2ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:31.412033 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6486d6-1642-4fd3-830a-1ca41693f2ba None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:31.430009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 454/901] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:31.432927 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350115a-fa1c-4223-a871-5b34f2afcd29 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:31.471395 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:31.546584 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350115a-fa1c-4223-a871-5b34f2afcd29 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:31.582606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aafd274-6a8c-47f7-877e-6ab018cb80f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:31.583670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafd274-6a8c-47f7-877e-6ab018cb80f7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:31.584145 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafd274-6a8c-47f7-877e-6ab018cb80f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:31.584565 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafd274-6a8c-47f7-877e-6ab018cb80f7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:31.592688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aafd274-6a8c-47f7-877e-6ab018cb80f7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:31.594308 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aafd274-6a8c-47f7-877e-6ab018cb80f7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:31.609600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-0aafd274-6a8c-47f7-877e-6ab018cb80f7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 10:25:31.609600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:31.609600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:31.609600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:31.609600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:31.609600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:31.609600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:31.609600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:31.609600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:31.609600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:31.609600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:31.609600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:31.609600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:25:31.609600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:25:31.609600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:25:31.609600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:25:31.610791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:31.610791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:31.610791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:31.610791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:31.610791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:31.610791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:31.610791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:31.610791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:31.610791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:31.610791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:31.610791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:31.610791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:31.610791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:31.610791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:31.610791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:31.610791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:31.610791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:31.610791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:31.612530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:31.612530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:31.612530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:25:31.612530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:25:31.612530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:25:31.612530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:25:31.612530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:25:31.612530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:25:31.612530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 10:25:31.612530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:31.614667 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 448/902] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jun 6 10:25:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:31.620440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6486d6-1642-4fd3-830a-1ca41693f2ba None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:31.733704 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6486d6-1642-4fd3-830a-1ca41693f2ba None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:31.760147 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:31.771140 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4588f2-e202-4005-a03a-1358cc7e6054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:31.772204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4588f2-e202-4005-a03a-1358cc7e6054 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:31.772718 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4588f2-e202-4005-a03a-1358cc7e6054 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:31.773451 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4588f2-e202-4005-a03a-1358cc7e6054 None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:31.774503 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4588f2-e202-4005-a03a-1358cc7e6054 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:31.775017 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4588f2-e202-4005-a03a-1358cc7e6054 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:31.776705 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4588f2-e202-4005-a03a-1358cc7e6054 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:31.795901 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 455/903] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Jun 6 10:25:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:31.804848 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aafd274-6a8c-47f7-877e-6ab018cb80f7 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:31.819235 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543a558-91b1-4356-9f11-aa8c94ae776e None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:31.819555 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543a558-91b1-4356-9f11-aa8c94ae776e None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:31.819842 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543a558-91b1-4356-9f11-aa8c94ae776e None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:31.821296 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 456/904] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:25:31.826991 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f427b6-900d-4e72-93a5-c6431847bad8 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:31.827538 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f427b6-900d-4e72-93a5-c6431847bad8 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:31.828071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f427b6-900d-4e72-93a5-c6431847bad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:31.847599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-24f427b6-900d-4e72-93a5-c6431847bad8 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:31.849728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-24f427b6-900d-4e72-93a5-c6431847bad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:31.909698 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:31.916705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aafd274-6a8c-47f7-877e-6ab018cb80f7 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:31.948218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 457/905] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:31 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 10:25:31.958089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2ab79-e4d4-4d2d-b937-8d0654474251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:31.959007 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2ab79-e4d4-4d2d-b937-8d0654474251 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:31.959492 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2ab79-e4d4-4d2d-b937-8d0654474251 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:31.960022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2ab79-e4d4-4d2d-b937-8d0654474251 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:31.972016 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f427b6-900d-4e72-93a5-c6431847bad8 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:32.042410 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2ab79-e4d4-4d2d-b937-8d0654474251 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:32.043664 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2ab79-e4d4-4d2d-b937-8d0654474251 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:32.102254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f427b6-900d-4e72-93a5-c6431847bad8 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:32.120375 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:32.146541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:32.156620 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 449/906] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:25:31 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 10:25:32.157950 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751dc82f-b216-4aee-9b3d-8f205fe685d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:32.158838 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dc82f-b216-4aee-9b3d-8f205fe685d2 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:32.159117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dc82f-b216-4aee-9b3d-8f205fe685d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:32.159393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dc82f-b216-4aee-9b3d-8f205fe685d2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:32.167837 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751dc82f-b216-4aee-9b3d-8f205fe685d2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:32.167837 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751dc82f-b216-4aee-9b3d-8f205fe685d2 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:32.168085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2ab79-e4d4-4d2d-b937-8d0654474251 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:32.178522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-751dc82f-b216-4aee-9b3d-8f205fe685d2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 10:25:32.178522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:32.178522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:32.178522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:32.178522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:32.178522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:32.178522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:32.178522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:32.178522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 10:25:32.178522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:32.178522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:32.178522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:32.178522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:32.178522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:32.178522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:32.178522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:32.179771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:32.179771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:32.179771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:32.179771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:32.179771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:32.179771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:32.179771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:32.179771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:32.179771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:32.179771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:32.179771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:32.179771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:32.179771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:32.179771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:32.179771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:32.179771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:32.181186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:32.181186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:32.181186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:32.181186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:32.181186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:32.181186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:32.181186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:32.181186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:32.181186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:32.181186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:32.181186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:32.181186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:32.181186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:32.181186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 10:25:32.181186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:32.181186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 458/907] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Jun 6 10:25:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:32.185930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751dc82f-b216-4aee-9b3d-8f205fe685d2 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:32.281391 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2ab79-e4d4-4d2d-b937-8d0654474251 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:32.298985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751dc82f-b216-4aee-9b3d-8f205fe685d2 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:32.320698 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb2dd31-999e-4012-8a0c-83c01ac4ddfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:32.321960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb2dd31-999e-4012-8a0c-83c01ac4ddfa None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:32.322434 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb2dd31-999e-4012-8a0c-83c01ac4ddfa None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:32.322913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb2dd31-999e-4012-8a0c-83c01ac4ddfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:32.334457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb2dd31-999e-4012-8a0c-83c01ac4ddfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:32.336481 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb2dd31-999e-4012-8a0c-83c01ac4ddfa None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:32.336618 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2a18d-2695-43dd-bfa9-26b68c198aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:32.337190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2a18d-2695-43dd-bfa9-26b68c198aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:32.337466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2a18d-2695-43dd-bfa9-26b68c198aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:32.337746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2a18d-2695-43dd-bfa9-26b68c198aa3 None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:32.338510 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea2a18d-2695-43dd-bfa9-26b68c198aa3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:32.338822 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea2a18d-2695-43dd-bfa9-26b68c198aa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:32.340535 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea2a18d-2695-43dd-bfa9-26b68c198aa3 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:32.350190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 459/908] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jun 6 10:25:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:32.351064 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2a18d-2695-43dd-bfa9-26b68c198aa3 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:32.358244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-9bb2dd31-999e-4012-8a0c-83c01ac4ddfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:32.358244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:32.358244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:32.358244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:32.358244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:32.358244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:32.358244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:32.358244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:32.358244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:32.358244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:32.358244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:32.358244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:32.358244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:32.358244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:32.358244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:32.359124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:32.359124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:32.359124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:32.359124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:32.359124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:32.359124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:32.359124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:32.359124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:32.359124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:32.359124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:32.359124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:32.359124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:32.359124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:32.359124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:32.359124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:32.359124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:32.359124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:32.360069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:32.360069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:32.360069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:32.360069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:32.367714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 450/909] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:32.369236 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb2dd31-999e-4012-8a0c-83c01ac4ddfa None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:32.393727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:32.431727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:32.462645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2a18d-2695-43dd-bfa9-26b68c198aa3 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:32.492183 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb2dd31-999e-4012-8a0c-83c01ac4ddfa None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:32.500885 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2615ab-afc6-4d09-8897-53f1f24a85ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:32.501935 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2615ab-afc6-4d09-8897-53f1f24a85ed None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:32.502405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2615ab-afc6-4d09-8897-53f1f24a85ed None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:32.502832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2615ab-afc6-4d09-8897-53f1f24a85ed None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:32.503701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2615ab-afc6-4d09-8897-53f1f24a85ed None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:32.504280 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2615ab-afc6-4d09-8897-53f1f24a85ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:32.505931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2615ab-afc6-4d09-8897-53f1f24a85ed None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:32.516601 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 460/910] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jun 6 10:25:32 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:32.517955 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2615ab-afc6-4d09-8897-53f1f24a85ed None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:32.532078 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba1947-e917-4cfb-9695-ee43fd066f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:32.533485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba1947-e917-4cfb-9695-ee43fd066f85 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:32.533643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba1947-e917-4cfb-9695-ee43fd066f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:32.533898 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba1947-e917-4cfb-9695-ee43fd066f85 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:32.534655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba1947-e917-4cfb-9695-ee43fd066f85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:32.534952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba1947-e917-4cfb-9695-ee43fd066f85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:32.536405 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba1947-e917-4cfb-9695-ee43fd066f85 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:32.555976 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 451/911] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:32.557503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba1947-e917-4cfb-9695-ee43fd066f85 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:32.636389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2615ab-afc6-4d09-8897-53f1f24a85ed None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:32.638435 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:32.672303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba1947-e917-4cfb-9695-ee43fd066f85 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:32.676054 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0699ecfe-dfbe-4fd7-997f-1da85d56178e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:32.676931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699ecfe-dfbe-4fd7-997f-1da85d56178e None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:32.677369 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699ecfe-dfbe-4fd7-997f-1da85d56178e None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:32.677807 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699ecfe-dfbe-4fd7-997f-1da85d56178e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:32.678697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0699ecfe-dfbe-4fd7-997f-1da85d56178e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:32.679891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:32.681398 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0699ecfe-dfbe-4fd7-997f-1da85d56178e None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:32.712362 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 461/912] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jun 6 10:25:32 2022] POST /identity/v3/endpoints => generated 347 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:25:32.717110 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb3864-b26f-4543-ada3-3e2c76b33838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:32.717933 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb3864-b26f-4543-ada3-3e2c76b33838 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:32.718216 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb3864-b26f-4543-ada3-3e2c76b33838 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:32.718489 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb3864-b26f-4543-ada3-3e2c76b33838 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:32.726707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb3864-b26f-4543-ada3-3e2c76b33838 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:32.728228 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb3864-b26f-4543-ada3-3e2c76b33838 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:32.744475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-28bb3864-b26f-4543-ada3-3e2c76b33838 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:32.744475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:32.744475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:32.744475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:32.744475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:32.744475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:32.744475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:32.744475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:32.744475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:32.744475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:32.744475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:32.744475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:32.744475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:32.744475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:32.744475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:32.745602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:32.745602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:32.745602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:32.745602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:32.745602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:32.745602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:32.745602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:32.745602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:32.745602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:32.745602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:32.745602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:32.745602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:32.745602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:32.745602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:32.745602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:32.745602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:32.745602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:32.746810 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:32.746810 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:32.746810 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:32.746810 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:32.746810 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:32.746810 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:32.746810 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:32.746810 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:32.750746 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 452/913] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:32.752835 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0699ecfe-dfbe-4fd7-997f-1da85d56178e None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:32.871356 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0699ecfe-dfbe-4fd7-997f-1da85d56178e None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:32.877575 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b68a3-b720-4da7-a39f-07e10e57fb7b None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:32.878081 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b68a3-b720-4da7-a39f-07e10e57fb7b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:32.878193 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b68a3-b720-4da7-a39f-07e10e57fb7b None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:32.883357 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 453/914] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:32.885196 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe45731-2f31-4afa-8433-8721be841cb7 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:32.885730 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe45731-2f31-4afa-8433-8721be841cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:32.886202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe45731-2f31-4afa-8433-8721be841cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:32.905168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-dfe45731-2f31-4afa-8433-8721be841cb7 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:32.906804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-dfe45731-2f31-4afa-8433-8721be841cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:32.911370 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7442fe-a018-4d70-a3b7-ff1e75addde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:32.912171 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7442fe-a018-4d70-a3b7-ff1e75addde4 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:32.912463 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7442fe-a018-4d70-a3b7-ff1e75addde4 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:32.912742 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7442fe-a018-4d70-a3b7-ff1e75addde4 None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:32.913598 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7442fe-a018-4d70-a3b7-ff1e75addde4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:32.913913 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7442fe-a018-4d70-a3b7-ff1e75addde4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:32.915375 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7442fe-a018-4d70-a3b7-ff1e75addde4 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:32.930530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 462/915] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:25:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:32.936934 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7442fe-a018-4d70-a3b7-ff1e75addde4 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:32.977598 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:33.010818 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 454/916] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:32 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:33.035480 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe45731-2f31-4afa-8433-8721be841cb7 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:33.079314 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7442fe-a018-4d70-a3b7-ff1e75addde4 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:33.120007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f9dec-5315-4be3-9228-8debc59d6de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:33.120706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f9dec-5315-4be3-9228-8debc59d6de3 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:33.121111 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f9dec-5315-4be3-9228-8debc59d6de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:33.121459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f9dec-5315-4be3-9228-8debc59d6de3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:33.122357 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f9dec-5315-4be3-9228-8debc59d6de3 None admin] RBAC: Adding query filter params (role.id=68d664d01d3340a58359d3fc9d738142, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=e9e358afe0974c0c8bd53104ebbff445) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:33.122740 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f9dec-5315-4be3-9228-8debc59d6de3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:33.124283 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f9dec-5315-4be3-9228-8debc59d6de3 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:33.135939 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 463/917] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:25:32 2022] GET /identity/v3/role_assignments?user.id=e9e358afe0974c0c8bd53104ebbff445&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=68d664d01d3340a58359d3fc9d738142 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:33.140182 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:33.144984 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe45731-2f31-4afa-8433-8721be841cb7 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:33.186201 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009683f-d76a-47a1-a90e-4eabaeb70db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:33.187056 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009683f-d76a-47a1-a90e-4eabaeb70db4 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:33.187340 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009683f-d76a-47a1-a90e-4eabaeb70db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:33.187614 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009683f-d76a-47a1-a90e-4eabaeb70db4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:33.188340 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009683f-d76a-47a1-a90e-4eabaeb70db4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:33.190002 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009683f-d76a-47a1-a90e-4eabaeb70db4 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:33.203814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-e009683f-d76a-47a1-a90e-4eabaeb70db4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 10:25:33.203814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:33.203814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:33.203814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:33.203814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:33.203814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:33.203814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:33.203814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:33.203814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 10:25:33.203814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:33.203814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:33.203814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:33.203814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:33.203814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:33.203814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:33.203814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:33.203814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:33.205914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:33.205914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:33.205914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:33.205914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:33.205914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:33.205914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:33.205914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:33.205914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:33.205914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:33.205914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:33.205914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:33.205914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:33.205914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:33.205914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:33.205914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:33.205914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:33.206879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:33.206879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:33.206879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:33.206879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:33.206879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:33.206879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:33.206879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:33.206879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:33.206879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:33.206879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:33.206879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:33.206879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:33.206879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 10:25:33.206879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:33.206879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 455/918] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:25:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:33.212200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f9dec-5315-4be3-9228-8debc59d6de3 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:33.315507 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:33.332806 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1369290-9ae5-4b6a-8213-7c0727aaa080 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:33.333104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1369290-9ae5-4b6a-8213-7c0727aaa080 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:33.333347 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1369290-9ae5-4b6a-8213-7c0727aaa080 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:33.335054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 456/919] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:25:33.342399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017af9fb-4614-4be5-a43e-57e3e1a0a0e9 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:33.342707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017af9fb-4614-4be5-a43e-57e3e1a0a0e9 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:33.342948 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017af9fb-4614-4be5-a43e-57e3e1a0a0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:33.348102 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f9dec-5315-4be3-9228-8debc59d6de3 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:33.361639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-017af9fb-4614-4be5-a43e-57e3e1a0a0e9 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:33.362942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-017af9fb-4614-4be5-a43e-57e3e1a0a0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:33.401670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbe037-6916-4d55-9fd7-08a6762adbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:33.402340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbe037-6916-4d55-9fd7-08a6762adbcc None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:33.402618 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbe037-6916-4d55-9fd7-08a6762adbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:33.402893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbe037-6916-4d55-9fd7-08a6762adbcc None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:33.403603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbe037-6916-4d55-9fd7-08a6762adbcc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:33.403941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbe037-6916-4d55-9fd7-08a6762adbcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:33.405311 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbe037-6916-4d55-9fd7-08a6762adbcc None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:33.412402 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 464/920] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:25:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:33.418245 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbe037-6916-4d55-9fd7-08a6762adbcc None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:33.451575 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 457/921] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 10:25:33.475206 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017af9fb-4614-4be5-a43e-57e3e1a0a0e9 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:33.550379 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbe037-6916-4d55-9fd7-08a6762adbcc None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:33.579432 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:33.590662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f21d6e-bce1-4bc5-aa33-13f5102e248d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:33.591757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f21d6e-bce1-4bc5-aa33-13f5102e248d None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:33.592190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f21d6e-bce1-4bc5-aa33-13f5102e248d None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:33.592470 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f21d6e-bce1-4bc5-aa33-13f5102e248d None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:33.593178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f21d6e-bce1-4bc5-aa33-13f5102e248d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:33.593489 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f21d6e-bce1-4bc5-aa33-13f5102e248d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:33.595901 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f21d6e-bce1-4bc5-aa33-13f5102e248d None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:33.598768 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017af9fb-4614-4be5-a43e-57e3e1a0a0e9 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:33.606968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 465/922] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jun 6 10:25:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:33.608013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f21d6e-bce1-4bc5-aa33-13f5102e248d None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:33.627253 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:33.643401 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c2975-7380-4c7c-887d-daa2c39d1192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:33.644599 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c2975-7380-4c7c-887d-daa2c39d1192 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:33.645269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c2975-7380-4c7c-887d-daa2c39d1192 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:33.645737 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c2975-7380-4c7c-887d-daa2c39d1192 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:33.646657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c2975-7380-4c7c-887d-daa2c39d1192 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:33.648723 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c2975-7380-4c7c-887d-daa2c39d1192 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:33.667900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 458/923] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jun 6 10:25:33 2022] POST /identity/v3/services => generated 241 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:25:33.744954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f21d6e-bce1-4bc5-aa33-13f5102e248d None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:33.791495 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef93921-fa47-4a5a-826a-ad40880068f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:33.792187 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef93921-fa47-4a5a-826a-ad40880068f6 None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:33.792469 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef93921-fa47-4a5a-826a-ad40880068f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:33.792741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef93921-fa47-4a5a-826a-ad40880068f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:33.793481 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef93921-fa47-4a5a-826a-ad40880068f6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:33.794914 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef93921-fa47-4a5a-826a-ad40880068f6 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:33.820957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 466/924] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jun 6 10:25:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:25:35.175472 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1e8cd-ad1b-4d52-b80e-1a22490744dd None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:35.176089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1e8cd-ad1b-4d52-b80e-1a22490744dd None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:35.176089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1e8cd-ad1b-4d52-b80e-1a22490744dd None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:35.177599 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 459/925] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:35.184143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32176a32-1061-40b2-8e10-b1526f5f5c2b None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:35.184667 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32176a32-1061-40b2-8e10-b1526f5f5c2b None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:35.184747 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32176a32-1061-40b2-8e10-b1526f5f5c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:35.202814 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-32176a32-1061-40b2-8e10-b1526f5f5c2b None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:35.204061 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-32176a32-1061-40b2-8e10-b1526f5f5c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:35.266731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:35.284719 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 467/926] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:35.307895 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1e8cd-ad1b-4d52-b80e-1a22490744dd None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:35.422605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1e8cd-ad1b-4d52-b80e-1a22490744dd None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:35.458682 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf04507-3e01-451c-a0db-3ae0dfc53684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:35.459736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf04507-3e01-451c-a0db-3ae0dfc53684 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:35.460213 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf04507-3e01-451c-a0db-3ae0dfc53684 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:35.460646 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf04507-3e01-451c-a0db-3ae0dfc53684 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:35.461436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf04507-3e01-451c-a0db-3ae0dfc53684 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:35.462948 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf04507-3e01-451c-a0db-3ae0dfc53684 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:35.476050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-1cf04507-3e01-451c-a0db-3ae0dfc53684 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 10:25:35.476050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:35.476050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:35.476050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:35.476050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:35.476050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:35.476050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:35.476050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:35.476050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 10:25:35.476050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:35.476050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:35.476050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:35.476050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:35.476050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:35.476050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:35.476050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:35.476050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:35.477262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:35.477262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:35.477262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:35.477262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:35.477262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:35.477262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:35.477262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:35.477262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:35.477262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:35.477262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:35.477262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:35.477262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:35.477262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:35.477262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:35.477262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:35.477262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:35.478304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:35.478304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:35.478304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:35.478304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:35.478304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:35.478304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:35.478304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:35.478304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:35.478304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:35.478304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:35.478304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:35.478304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:35.478304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 10:25:35.478304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:35.480772 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 460/927] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:35.487243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32176a32-1061-40b2-8e10-b1526f5f5c2b None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:35.570110 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e72b0-577b-466c-a2b4-9007a0cc18dc None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:35.570401 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e72b0-577b-466c-a2b4-9007a0cc18dc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:35.570685 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e72b0-577b-466c-a2b4-9007a0cc18dc None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:35.572452 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 461/928] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:35.577489 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e37ac-a882-42f0-951b-c96c8f00cd7b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:35.577870 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e37ac-a882-42f0-951b-c96c8f00cd7b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:35.578216 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e37ac-a882-42f0-951b-c96c8f00cd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:35.592738 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32176a32-1061-40b2-8e10-b1526f5f5c2b None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:35.597255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-e45e37ac-a882-42f0-951b-c96c8f00cd7b None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:35.598561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e45e37ac-a882-42f0-951b-c96c8f00cd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:35.630422 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d8d266-3abf-4fba-b5fd-31721918c152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:35.631076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:35.631189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8d266-3abf-4fba-b5fd-31721918c152 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:35.631349 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8d266-3abf-4fba-b5fd-31721918c152 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:35.631620 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8d266-3abf-4fba-b5fd-31721918c152 None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:35.632394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d8d266-3abf-4fba-b5fd-31721918c152 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:35.632706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d8d266-3abf-4fba-b5fd-31721918c152 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:35.634118 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d8d266-3abf-4fba-b5fd-31721918c152 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:35.640087 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 468/929] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jun 6 10:25:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:35.646929 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d8d266-3abf-4fba-b5fd-31721918c152 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:35.687973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 462/930] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:35.704102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e37ac-a882-42f0-951b-c96c8f00cd7b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:35.707710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:35.763426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d8d266-3abf-4fba-b5fd-31721918c152 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:35.802254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e380b40-7724-47d0-af34-55ac39bb98f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:35.802901 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e380b40-7724-47d0-af34-55ac39bb98f7 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:35.803178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e380b40-7724-47d0-af34-55ac39bb98f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:35.803454 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e380b40-7724-47d0-af34-55ac39bb98f7 None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:35.804186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e380b40-7724-47d0-af34-55ac39bb98f7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:35.804490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e380b40-7724-47d0-af34-55ac39bb98f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:35.805990 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e380b40-7724-47d0-af34-55ac39bb98f7 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:35.811172 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 469/931] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jun 6 10:25:35 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:35.819406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e37ac-a882-42f0-951b-c96c8f00cd7b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:35.864663 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94d296-c272-4b36-98d6-7a82fca70a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:35.865854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94d296-c272-4b36-98d6-7a82fca70a6d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:35.866358 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94d296-c272-4b36-98d6-7a82fca70a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:35.866831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94d296-c272-4b36-98d6-7a82fca70a6d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:35.878113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:35.881642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94d296-c272-4b36-98d6-7a82fca70a6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:25:35.890755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-bf94d296-c272-4b36-98d6-7a82fca70a6d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 10:25:35.890755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:35.890755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:35.890755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:35.890755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:35.890755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:35.890755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:35.890755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:35.890755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:35.890755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:35.890755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:35.890755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:35.890755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:25:35.890755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:25:35.890755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:25:35.890755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:25:35.894319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:25:35.894319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:25:35.894319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:25:35.894319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:25:35.894319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:35.894319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:35.894319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:35.894319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:35.894319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:35.894319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:35.894319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:35.894319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:35.894319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:35.894319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:35.894319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:35.894319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:35.895158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:35.895158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:35.895158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:35.895158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:35.895158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:25:35.895158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:25:35.895158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:25:35.895158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:25:35.895158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:25:35.895158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:25:35.895158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 10:25:35.895158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:35.895158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 463/932] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Jun 6 10:25:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:35.899222 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e380b40-7724-47d0-af34-55ac39bb98f7 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:36.015396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e380b40-7724-47d0-af34-55ac39bb98f7 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:36.035444 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:36.056316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030e7b7-db77-4103-a198-10ba1bc61de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:36.056991 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030e7b7-db77-4103-a198-10ba1bc61de2 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:36.057275 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030e7b7-db77-4103-a198-10ba1bc61de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:36.057550 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030e7b7-db77-4103-a198-10ba1bc61de2 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:36.058307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9030e7b7-db77-4103-a198-10ba1bc61de2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:36.058800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9030e7b7-db77-4103-a198-10ba1bc61de2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:36.060290 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb88ec3-4622-455b-8f03-83fb7145493c None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:36.060575 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb88ec3-4622-455b-8f03-83fb7145493c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:36.060843 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9030e7b7-db77-4103-a198-10ba1bc61de2 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:36.060941 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb88ec3-4622-455b-8f03-83fb7145493c None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:36.062448 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 464/933] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:36.067584 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55e567-4675-4957-9814-50fdc9167cfe None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:36.067911 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55e567-4675-4957-9814-50fdc9167cfe None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:36.068182 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55e567-4675-4957-9814-50fdc9167cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:36.070540 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 470/934] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Jun 6 10:25:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:36.076034 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030e7b7-db77-4103-a198-10ba1bc61de2 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:36.090603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-6d55e567-4675-4957-9814-50fdc9167cfe None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:36.092185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6d55e567-4675-4957-9814-50fdc9167cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:36.177121 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 465/935] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 10:25:36.197501 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030e7b7-db77-4103-a198-10ba1bc61de2 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:36.201925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55e567-4675-4957-9814-50fdc9167cfe None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:36.236963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a704f-1dd3-46d4-bf3b-000ebf4a1776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:36.237810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a704f-1dd3-46d4-bf3b-000ebf4a1776 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:36.238142 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a704f-1dd3-46d4-bf3b-000ebf4a1776 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:36.238417 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a704f-1dd3-46d4-bf3b-000ebf4a1776 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:36.249884 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a704f-1dd3-46d4-bf3b-000ebf4a1776 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:36.251478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a704f-1dd3-46d4-bf3b-000ebf4a1776 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:36.269359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-2c9a704f-1dd3-46d4-bf3b-000ebf4a1776 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:36.269359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:36.269359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:36.269359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:36.269359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:36.269359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:36.269359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:36.269359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:36.269359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:36.269359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:36.269359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:36.269359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:36.269359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:36.269359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:36.269359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:36.270265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:36.270265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:36.270265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:36.270265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:36.270265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:36.270265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:36.270265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:36.270265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:36.270265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:36.270265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:36.270265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:36.270265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:36.270265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:36.270265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:36.270265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:36.270265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:36.270265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:36.271271 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:36.271271 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:36.271271 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:36.271271 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:36.271271 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 471/936] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:36.272230 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d603c45-81f7-480c-9dba-3c9b43a70ed3 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:36.272521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d603c45-81f7-480c-9dba-3c9b43a70ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:36.272799 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d603c45-81f7-480c-9dba-3c9b43a70ed3 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:36.275856 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 472/937] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:36.276121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d603c45-81f7-480c-9dba-3c9b43a70ed3 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:36.309610 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:36.316277 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55e567-4675-4957-9814-50fdc9167cfe None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:36.347856 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:36.355711 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4aa697-e0fd-4b23-abdd-276d8e3c992c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:36.356580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4aa697-e0fd-4b23-abdd-276d8e3c992c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:36.356862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4aa697-e0fd-4b23-abdd-276d8e3c992c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:36.357132 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4aa697-e0fd-4b23-abdd-276d8e3c992c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:36.357783 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4aa697-e0fd-4b23-abdd-276d8e3c992c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:36.359149 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4aa697-e0fd-4b23-abdd-276d8e3c992c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:36.372844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-dd4aa697-e0fd-4b23-abdd-276d8e3c992c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 10:25:36.372844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:36.372844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:36.372844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:36.372844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:36.372844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:36.372844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:36.372844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:36.372844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 10:25:36.372844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:36.372844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:36.372844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:36.372844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:36.372844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:36.372844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:36.372844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:36.372844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:36.373891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:36.373891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:36.373891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:36.373891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:36.373891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:36.373891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:36.373891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:36.373891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:36.373891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:36.373891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:36.373891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:36.373891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:36.373891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:36.373891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:36.373891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:36.373891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:36.374771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:36.374771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:36.374771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:36.374771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:36.374771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:36.374771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:36.374771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:36.374771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:36.374771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:36.374771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:36.374771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:36.374771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:36.374771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 10:25:36.374771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:36.374771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 466/938] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jun 6 10:25:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:36.375587 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b58271-a4de-4ad8-ad0a-cd25cc9794a6 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:36.375587 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b58271-a4de-4ad8-ad0a-cd25cc9794a6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:36.376949 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b58271-a4de-4ad8-ad0a-cd25cc9794a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:36.394849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d603c45-81f7-480c-9dba-3c9b43a70ed3 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:36.396727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b0b58271-a4de-4ad8-ad0a-cd25cc9794a6 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:36.398154 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b0b58271-a4de-4ad8-ad0a-cd25cc9794a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:36.436750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70f21a-96c7-4068-acf7-01fe6328fd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:36.437413 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70f21a-96c7-4068-acf7-01fe6328fd3b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:36.437697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70f21a-96c7-4068-acf7-01fe6328fd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:36.437971 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70f21a-96c7-4068-acf7-01fe6328fd3b None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:36.438695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70f21a-96c7-4068-acf7-01fe6328fd3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:36.439151 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70f21a-96c7-4068-acf7-01fe6328fd3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:36.440682 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70f21a-96c7-4068-acf7-01fe6328fd3b None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:36.453264 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 473/939] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:36.454049 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70f21a-96c7-4068-acf7-01fe6328fd3b None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:36.475825 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 467/940] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:36.475825 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b58271-a4de-4ad8-ad0a-cd25cc9794a6 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:36.574860 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b58271-a4de-4ad8-ad0a-cd25cc9794a6 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:36.575045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:36.586453 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70f21a-96c7-4068-acf7-01fe6328fd3b None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:36.605114 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:36.620362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a622fe1-1f45-49a0-9c7a-aa1fcf1ec06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:36.620738 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927380aa-691e-45b3-a689-4e01084d5788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:36.621399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927380aa-691e-45b3-a689-4e01084d5788 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:36.621679 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927380aa-691e-45b3-a689-4e01084d5788 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:36.621893 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a622fe1-1f45-49a0-9c7a-aa1fcf1ec06b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:36.622009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927380aa-691e-45b3-a689-4e01084d5788 None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:36.622406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a622fe1-1f45-49a0-9c7a-aa1fcf1ec06b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:36.622631 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927380aa-691e-45b3-a689-4e01084d5788 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:36.622933 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927380aa-691e-45b3-a689-4e01084d5788 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:36.623113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a622fe1-1f45-49a0-9c7a-aa1fcf1ec06b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:36.624327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927380aa-691e-45b3-a689-4e01084d5788 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:36.634702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 474/941] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jun 6 10:25:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:36.634702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927380aa-691e-45b3-a689-4e01084d5788 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:36.634984 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a622fe1-1f45-49a0-9c7a-aa1fcf1ec06b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:36.635751 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a622fe1-1f45-49a0-9c7a-aa1fcf1ec06b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:36.651436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-9a622fe1-1f45-49a0-9c7a-aa1fcf1ec06b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 10:25:36.651436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:36.651436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:36.651436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:36.651436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:36.651436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:36.651436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:36.651436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:36.651436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:36.651436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:36.651436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:36.651436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:36.651436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:25:36.651436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:25:36.651436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:25:36.651436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:25:36.652627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:36.652627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:36.652627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:36.652627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:36.652627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:36.652627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:36.652627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:36.652627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:36.652627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:36.652627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:36.652627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:36.652627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:36.652627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:36.652627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:36.652627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:36.652627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:36.652627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:36.652627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:36.653804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:36.653804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:36.653804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:25:36.653804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:25:36.653804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:25:36.653804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:25:36.653804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:25:36.653804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:25:36.653804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 10:25:36.653804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:36.659976 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 468/942] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jun 6 10:25:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:36.659976 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a622fe1-1f45-49a0-9c7a-aa1fcf1ec06b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:36.748996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927380aa-691e-45b3-a689-4e01084d5788 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:36.759452 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a622fe1-1f45-49a0-9c7a-aa1fcf1ec06b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:36.786467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d518abe-3d83-419f-9bf2-4559e8ac03d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:36.787489 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d518abe-3d83-419f-9bf2-4559e8ac03d8 None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:36.788072 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d518abe-3d83-419f-9bf2-4559e8ac03d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:36.788510 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d518abe-3d83-419f-9bf2-4559e8ac03d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:36.789352 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d518abe-3d83-419f-9bf2-4559e8ac03d8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:36.790930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d518abe-3d83-419f-9bf2-4559e8ac03d8 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:36.792478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: WARNING keystone.server.flask.common [None req-4d518abe-3d83-419f-9bf2-4559e8ac03d8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 06 10:25:36.792599 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e3913-4d04-4775-aa2f-12140361774d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:36.793265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e3913-4d04-4775-aa2f-12140361774d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:36.793551 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e3913-4d04-4775-aa2f-12140361774d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:36.793830 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e3913-4d04-4775-aa2f-12140361774d None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:36.794540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e3913-4d04-4775-aa2f-12140361774d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:36.794846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e3913-4d04-4775-aa2f-12140361774d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:36.796225 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e3913-4d04-4775-aa2f-12140361774d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:36.801432 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 469/943] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jun 6 10:25:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:36.802201 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e3913-4d04-4775-aa2f-12140361774d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:36.817261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.sql.core [None req-4d518abe-3d83-419f-9bf2-4559e8ac03d8 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 06 10:25:36.817261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 06 10:25:36.817261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [parameters: {'id': 'e273ff08f2434679acad15912b6c713b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 06 10:25:36.817261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80241) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 06 10:25:36.835581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-4d518abe-3d83-419f-9bf2-4559e8ac03d8 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e273ff08f2434679acad15912b6c713b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e273ff08f2434679acad15912b6c713b. Jun 06 10:25:36.835581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:36.835581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 10:25:36.835581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 10:25:36.835581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 10:25:36.835581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 10:25:36.835581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 10:25:36.835581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 10:25:36.835581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 10:25:36.835581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application conn.query(q) Jun 06 10:25:36.835581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 10:25:36.835581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 10:25:36.835581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 10:25:36.836585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application result.read() Jun 06 10:25:36.836585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 10:25:36.836585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 10:25:36.836585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 10:25:36.836585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 10:25:36.836585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 10:25:36.836585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 10:25:36.836585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 10:25:36.836585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 10:25:36.836585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 06 10:25:36.836585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:36.836585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 06 10:25:36.836585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:36.836585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:36.836585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 06 10:25:36.836585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 06 10:25:36.836585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 06 10:25:36.836585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 06 10:25:36.837750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 10:25:36.837750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application next(self.gen) Jun 06 10:25:36.837750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 06 10:25:36.837750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application yield resource Jun 06 10:25:36.837750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 10:25:36.837750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application next(self.gen) Jun 06 10:25:36.837750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 06 10:25:36.837750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self.session.rollback() Jun 06 10:25:36.837750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 06 10:25:36.837750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self.force_reraise() Jun 06 10:25:36.837750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 06 10:25:36.837750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise self.value Jun 06 10:25:36.837750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 06 10:25:36.837750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 06 10:25:36.837750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 06 10:25:36.837750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application session.commit() Jun 06 10:25:36.837750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 06 10:25:36.838849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 06 10:25:36.838849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 06 10:25:36.838849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._prepare_impl() Jun 06 10:25:36.838849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 06 10:25:36.838849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self.session.flush() Jun 06 10:25:36.838849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 06 10:25:36.838849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._flush(objects) Jun 06 10:25:36.838849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 06 10:25:36.838849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 06 10:25:36.838849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 06 10:25:36.838849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application compat.raise_( Jun 06 10:25:36.838849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 10:25:36.838849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception Jun 06 10:25:36.838849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 06 10:25:36.838849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application flush_context.execute() Jun 06 10:25:36.838849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 06 10:25:36.838849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rec.execute(self) Jun 06 10:25:36.840228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 06 10:25:36.840228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 06 10:25:36.840228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 06 10:25:36.840228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 06 10:25:36.840228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 06 10:25:36.840228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 06 10:25:36.840228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 06 10:25:36.840228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 06 10:25:36.840228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 06 10:25:36.840228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 06 10:25:36.840228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 06 10:25:36.840228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 06 10:25:36.840228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 06 10:25:36.840228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 06 10:25:36.840228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 06 10:25:36.841493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 06 10:25:36.841493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 10:25:36.841493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception Jun 06 10:25:36.841493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 10:25:36.841493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 10:25:36.841493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 10:25:36.841493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 10:25:36.841493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 10:25:36.841493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 10:25:36.841493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 10:25:36.841493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application conn.query(q) Jun 06 10:25:36.841493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 10:25:36.841493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 10:25:36.841493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 10:25:36.841493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application result.read() Jun 06 10:25:36.841493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 10:25:36.841493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 10:25:36.842586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 10:25:36.842586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 10:25:36.842586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 10:25:36.842586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 10:25:36.842586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 10:25:36.842586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 10:25:36.842586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 06 10:25:36.842586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 06 10:25:36.842586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [parameters: {'id': 'e273ff08f2434679acad15912b6c713b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 06 10:25:36.842586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 06 10:25:36.842586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:36.842586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:36.842586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:36.842586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:36.842586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 06 10:25:36.843568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 06 10:25:36.843568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 06 10:25:36.843568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 06 10:25:36.843568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 06 10:25:36.843568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:36.843568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:36.843568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:36.843568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:36.843568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:36.843568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:36.843568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:36.843568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:36.843568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:36.843568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:36.843568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:36.843568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:36.843568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:36.844714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:36.844714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 06 10:25:36.844714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 06 10:25:36.844714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:36.844714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:36.844714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 06 10:25:36.844714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 06 10:25:36.844714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e273ff08f2434679acad15912b6c713b. Jun 06 10:25:36.844714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:36.848383 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 475/944] 10.222.0.42 () {60 vars in 1140 bytes} [Mon Jun 6 10:25:36 2022] POST /identity/v3/projects => generated 262 bytes in 216 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:36.848383 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d518abe-3d83-419f-9bf2-4559e8ac03d8 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:36.863183 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:36.868386 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:36.918572 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e3913-4d04-4775-aa2f-12140361774d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:36.956851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5a696-8ccd-44e2-ba52-17c19cc4a1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:36.957595 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5a696-8ccd-44e2-ba52-17c19cc4a1a2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:36.957875 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5a696-8ccd-44e2-ba52-17c19cc4a1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:36.958142 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5a696-8ccd-44e2-ba52-17c19cc4a1a2 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:36.959228 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5a696-8ccd-44e2-ba52-17c19cc4a1a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:36.959568 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5a696-8ccd-44e2-ba52-17c19cc4a1a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:36.961022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5a696-8ccd-44e2-ba52-17c19cc4a1a2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:36.966081 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d518abe-3d83-419f-9bf2-4559e8ac03d8 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:36.986494 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 470/945] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Jun 6 10:25:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:36.986494 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5a696-8ccd-44e2-ba52-17c19cc4a1a2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:37.010340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858f3b8-c28e-4084-a968-3f8bdfe149b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:37.011018 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858f3b8-c28e-4084-a968-3f8bdfe149b3 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:37.011307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858f3b8-c28e-4084-a968-3f8bdfe149b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:37.011587 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858f3b8-c28e-4084-a968-3f8bdfe149b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:37.012429 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858f3b8-c28e-4084-a968-3f8bdfe149b3 None admin] RBAC: Adding query filter params (interface=public, service_id=cc2623eebf484287b63e512e59444110, region_id=RegionOne) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:37.012752 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858f3b8-c28e-4084-a968-3f8bdfe149b3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:37.014378 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858f3b8-c28e-4084-a968-3f8bdfe149b3 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:37.024003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 476/946] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jun 6 10:25:36 2022] GET /identity/v3/endpoints?service_id=cc2623eebf484287b63e512e59444110&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:37.024864 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858f3b8-c28e-4084-a968-3f8bdfe149b3 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:37.093900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5a696-8ccd-44e2-ba52-17c19cc4a1a2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:37.102843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:37.111080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:37.135416 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cb25d-e7d6-4413-80e2-f6eb7d63156e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:37.136478 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cb25d-e7d6-4413-80e2-f6eb7d63156e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:37.136761 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cb25d-e7d6-4413-80e2-f6eb7d63156e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:37.137040 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cb25d-e7d6-4413-80e2-f6eb7d63156e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:37.139146 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858f3b8-c28e-4084-a968-3f8bdfe149b3 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:37.144684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cb25d-e7d6-4413-80e2-f6eb7d63156e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:37.146131 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cb25d-e7d6-4413-80e2-f6eb7d63156e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:37.163356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-526cb25d-e7d6-4413-80e2-f6eb7d63156e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:37.163356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:37.163356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:37.163356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:37.163356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:37.163356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:37.163356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:37.163356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:37.163356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:37.163356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:37.163356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:37.163356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:37.163356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:37.163356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:37.163356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:37.164318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:37.164318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:37.164318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:37.164318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:37.164318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:37.164318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:37.164318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:37.164318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:37.164318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:37.164318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:37.164318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:37.164318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:37.164318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:37.164318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:37.164318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:37.164318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:37.164318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:37.165187 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:37.165187 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:37.165187 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:37.165187 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:37.165187 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:37.165187 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:37.165187 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:37.165187 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:37.165187 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 471/947] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:37.166258 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cb25d-e7d6-4413-80e2-f6eb7d63156e None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:37.183623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d302a67-6508-46c4-88ff-b9252b0dd1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:37.184449 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d302a67-6508-46c4-88ff-b9252b0dd1be None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:37.184733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d302a67-6508-46c4-88ff-b9252b0dd1be None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:37.185004 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d302a67-6508-46c4-88ff-b9252b0dd1be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:37.196995 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d302a67-6508-46c4-88ff-b9252b0dd1be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:37.198628 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d302a67-6508-46c4-88ff-b9252b0dd1be None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:37.216703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-7d302a67-6508-46c4-88ff-b9252b0dd1be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:37.216703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:37.216703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:37.216703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:37.216703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:37.216703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:37.216703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:37.216703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:37.216703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:37.216703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:37.216703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:37.216703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:37.216703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:37.216703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:37.216703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:37.217576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:37.217576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:37.217576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:37.217576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:37.217576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:37.217576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:37.217576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:37.217576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:37.217576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:37.217576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:37.217576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:37.217576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:37.217576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:37.217576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:37.217576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:37.217576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:37.217576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:37.218457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:37.218457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:37.218457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:37.218457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:37.218457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 477/948] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:37.218732 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d302a67-6508-46c4-88ff-b9252b0dd1be None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:37.278842 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cb25d-e7d6-4413-80e2-f6eb7d63156e None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:37.315512 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94562a93-5bb0-4bec-aa1a-78463ba2b56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:37.316201 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94562a93-5bb0-4bec-aa1a-78463ba2b56a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:37.316597 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94562a93-5bb0-4bec-aa1a-78463ba2b56a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:37.316872 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94562a93-5bb0-4bec-aa1a-78463ba2b56a None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:37.317671 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94562a93-5bb0-4bec-aa1a-78463ba2b56a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:37.319027 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94562a93-5bb0-4bec-aa1a-78463ba2b56a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:37.322655 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d302a67-6508-46c4-88ff-b9252b0dd1be None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:37.335830 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 472/949] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jun 6 10:25:37 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:25:37.335830 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94562a93-5bb0-4bec-aa1a-78463ba2b56a None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:37.357949 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:37.369827 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf2623-af0a-4bbd-9c58-235550d4d32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:37.370723 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf2623-af0a-4bbd-9c58-235550d4d32b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:37.371105 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf2623-af0a-4bbd-9c58-235550d4d32b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:37.371490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf2623-af0a-4bbd-9c58-235550d4d32b None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:37.372382 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf2623-af0a-4bbd-9c58-235550d4d32b None admin] RBAC: Adding query filter params (name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:37.372790 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf2623-af0a-4bbd-9c58-235550d4d32b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:37.374246 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:37.374436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf2623-af0a-4bbd-9c58-235550d4d32b None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:37.406640 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 478/950] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jun 6 10:25:37 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:37.474543 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94562a93-5bb0-4bec-aa1a-78463ba2b56a None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:37.515692 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e55a7-7dc1-468f-9325-c3718a58ba39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:37.516454 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e55a7-7dc1-468f-9325-c3718a58ba39 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:37.516732 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e55a7-7dc1-468f-9325-c3718a58ba39 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:37.517567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e55a7-7dc1-468f-9325-c3718a58ba39 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:37.518856 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e55a7-7dc1-468f-9325-c3718a58ba39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:37.519168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e55a7-7dc1-468f-9325-c3718a58ba39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:37.520697 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e55a7-7dc1-468f-9325-c3718a58ba39 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:37.536074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 473/951] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:37.541713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf2623-af0a-4bbd-9c58-235550d4d32b None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:37.662754 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf2623-af0a-4bbd-9c58-235550d4d32b None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:37.698979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a88bd1-6440-498b-8e81-57a37da0350b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:37.700172 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a88bd1-6440-498b-8e81-57a37da0350b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:37.700679 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a88bd1-6440-498b-8e81-57a37da0350b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:37.701165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a88bd1-6440-498b-8e81-57a37da0350b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:37.709619 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a88bd1-6440-498b-8e81-57a37da0350b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:37.711429 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a88bd1-6440-498b-8e81-57a37da0350b None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:37.726559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-f0a88bd1-6440-498b-8e81-57a37da0350b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:37.726559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:37.726559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:37.726559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:37.726559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:37.726559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:37.726559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:37.726559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:37.726559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:37.726559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:37.726559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:37.726559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:37.726559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:37.726559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:37.726559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:37.727581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:37.727581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:37.727581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:37.727581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:37.727581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:37.727581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:37.727581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:37.727581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:37.727581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:37.727581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:37.727581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:37.727581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:37.727581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:37.727581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:37.727581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:37.727581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:37.727581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:37.728751 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:37.728751 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:37.728751 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:37.728751 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:37.728751 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:37.728751 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:37.728751 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:37.728751 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:37.729192 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 479/952] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:37.737030 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e55a7-7dc1-468f-9325-c3718a58ba39 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:37.834591 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:37.845039 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e55a7-7dc1-468f-9325-c3718a58ba39 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:37.884537 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3d84a-bcc4-4e4c-8d53-19c4e54e4e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:37.885201 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3d84a-bcc4-4e4c-8d53-19c4e54e4e2c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:37.885481 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3d84a-bcc4-4e4c-8d53-19c4e54e4e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:37.885753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3d84a-bcc4-4e4c-8d53-19c4e54e4e2c None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:37.886532 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3d84a-bcc4-4e4c-8d53-19c4e54e4e2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:37.886834 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3d84a-bcc4-4e4c-8d53-19c4e54e4e2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:37.888298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3d84a-bcc4-4e4c-8d53-19c4e54e4e2c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:37.901555 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 474/953] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:25:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:37.907373 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a88bd1-6440-498b-8e81-57a37da0350b None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:37.968972 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:38.023198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a88bd1-6440-498b-8e81-57a37da0350b None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:38.067969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77834a60-eb61-498d-9b5c-5abac0fd2a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:38.069145 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77834a60-eb61-498d-9b5c-5abac0fd2a34 None admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:38.069574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77834a60-eb61-498d-9b5c-5abac0fd2a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:38.070017 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77834a60-eb61-498d-9b5c-5abac0fd2a34 None admin] PATH_INFO: `/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/e9e358afe0974c0c8bd53104ebbff445/roles/68d664d01d3340a58359d3fc9d738142` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:38.072683 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77834a60-eb61-498d-9b5c-5abac0fd2a34 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f082d7ed3694fd9998623f3d4a5fea9, user_id=e9e358afe0974c0c8bd53104ebbff445, role_id=68d664d01d3340a58359d3fc9d738142)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:38.074210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77834a60-eb61-498d-9b5c-5abac0fd2a34 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:38.092899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 480/954] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:25:37 2022] PUT /identity/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/e9e358afe0974c0c8bd53104ebbff445/roles/68d664d01d3340a58359d3fc9d738142 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:25:38.203402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6e11b-483d-4c6d-917b-0d4981dbe612 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:38.203686 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6e11b-483d-4c6d-917b-0d4981dbe612 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:38.203999 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6e11b-483d-4c6d-917b-0d4981dbe612 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:38.206372 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 475/955] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:38.212394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2641e5-9b84-4cb0-a2af-5e2f0a955edc None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:38.212904 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2641e5-9b84-4cb0-a2af-5e2f0a955edc None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:38.213016 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2641e5-9b84-4cb0-a2af-5e2f0a955edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:38.232865 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-fc2641e5-9b84-4cb0-a2af-5e2f0a955edc None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:38.234345 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-fc2641e5-9b84-4cb0-a2af-5e2f0a955edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:38.318903 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 481/956] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:38.342885 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6e11b-483d-4c6d-917b-0d4981dbe612 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:38.448705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6e11b-483d-4c6d-917b-0d4981dbe612 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:38.483709 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6488ff7c-22a6-4d13-b0c8-0f39ba8431ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:38.484335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488ff7c-22a6-4d13-b0c8-0f39ba8431ba None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:38.484620 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488ff7c-22a6-4d13-b0c8-0f39ba8431ba None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:38.484897 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488ff7c-22a6-4d13-b0c8-0f39ba8431ba None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:38.485662 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6488ff7c-22a6-4d13-b0c8-0f39ba8431ba None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:38.487147 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6488ff7c-22a6-4d13-b0c8-0f39ba8431ba None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:38.507517 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 476/957] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jun 6 10:25:38 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:25:39.806737 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146644fe-8a5c-4872-96ef-24925427d84e None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:39.807730 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146644fe-8a5c-4872-96ef-24925427d84e None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:39.808248 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146644fe-8a5c-4872-96ef-24925427d84e None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:39.810085 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 482/958] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:39.816471 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2b2fa-f86e-4579-8d06-c42f43244d09 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:39.817300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2b2fa-f86e-4579-8d06-c42f43244d09 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:39.817793 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2b2fa-f86e-4579-8d06-c42f43244d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:39.835855 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-04a2b2fa-f86e-4579-8d06-c42f43244d09 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:39.837436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-04a2b2fa-f86e-4579-8d06-c42f43244d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:39.840634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:39.860321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0e48d-c06c-4979-b033-5b56ce757b0b None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:39.861133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0e48d-c06c-4979-b033-5b56ce757b0b None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:39.861563 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0e48d-c06c-4979-b033-5b56ce757b0b None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:39.863191 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 483/959] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:25:39.868085 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfbe2c-618f-411b-a33a-7b6ef3977114 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:39.868373 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfbe2c-618f-411b-a33a-7b6ef3977114 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:39.868649 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfbe2c-618f-411b-a33a-7b6ef3977114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:39.887035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-d2bfbe2c-618f-411b-a33a-7b6ef3977114 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:39.888249 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-d2bfbe2c-618f-411b-a33a-7b6ef3977114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:39.922860 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 477/960] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:39 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:39.923894 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:39.950618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2b2fa-f86e-4579-8d06-c42f43244d09 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:39.974235 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 484/961] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:39 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jun 06 10:25:39.999304 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bfbe2c-618f-411b-a33a-7b6ef3977114 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:40.055272 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2b2fa-f86e-4579-8d06-c42f43244d09 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:40.092201 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d30d2-9041-4f2b-ad5b-faad27f06ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:40.092930 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d30d2-9041-4f2b-ad5b-faad27f06ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:40.093215 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d30d2-9041-4f2b-ad5b-faad27f06ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:40.093493 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d30d2-9041-4f2b-ad5b-faad27f06ee3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:40.094149 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d30d2-9041-4f2b-ad5b-faad27f06ee3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:40.095895 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d30d2-9041-4f2b-ad5b-faad27f06ee3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:40.098795 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:40.109005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bfbe2c-618f-411b-a33a-7b6ef3977114 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:40.110831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-0f5d30d2-9041-4f2b-ad5b-faad27f06ee3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 10:25:40.110831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:40.110831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:40.110831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:40.110831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:40.110831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:40.110831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:40.110831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:40.110831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 10:25:40.110831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:40.110831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:40.110831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:40.110831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:40.110831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:40.110831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:40.110831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:40.110831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:40.111946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:40.111946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:40.111946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:40.111946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:40.111946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:40.111946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:40.111946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:40.111946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:40.111946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:40.111946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:40.111946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:40.111946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:40.111946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:40.111946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:40.111946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:40.111946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:40.112816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:40.112816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:40.112816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:40.112816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:40.112816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:40.112816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:40.112816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:40.112816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:40.112816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:40.112816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:40.112816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:40.112816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:40.112816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 10:25:40.112816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:40.112816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 478/962] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jun 6 10:25:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:40.118686 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d30d2-9041-4f2b-ad5b-faad27f06ee3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:40.155824 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb30af-d2ae-4831-84d2-23765321dd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:40.156874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb30af-d2ae-4831-84d2-23765321dd95 None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:40.157393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb30af-d2ae-4831-84d2-23765321dd95 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:40.157936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb30af-d2ae-4831-84d2-23765321dd95 None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:40.159036 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb30af-d2ae-4831-84d2-23765321dd95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:40.160697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb30af-d2ae-4831-84d2-23765321dd95 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:40.174595 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.sql.core [None req-7acb30af-d2ae-4831-84d2-23765321dd95 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 06 10:25:40.174595 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 06 10:25:40.174595 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [parameters: {'user_id': '048c6896f68a4d62befbb70e243ce40b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 06 10:25:40.174595 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80241) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 06 10:25:40.186063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-7acb30af-d2ae-4831-84d2-23765321dd95 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 06 10:25:40.186063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:40.186063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 10:25:40.186063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 10:25:40.186063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 10:25:40.186063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 10:25:40.186063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 10:25:40.186063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 10:25:40.186063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 10:25:40.186063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application conn.query(q) Jun 06 10:25:40.186063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 10:25:40.186063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 10:25:40.186063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 10:25:40.186063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application result.read() Jun 06 10:25:40.186063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 10:25:40.186947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 10:25:40.186947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 10:25:40.186947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 10:25:40.186947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 10:25:40.186947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 10:25:40.186947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 10:25:40.186947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 10:25:40.186947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 06 10:25:40.186947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:40.186947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 06 10:25:40.186947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:40.186947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:40.186947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 06 10:25:40.186947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 06 10:25:40.186947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 06 10:25:40.186947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 06 10:25:40.186947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 10:25:40.186947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application next(self.gen) Jun 06 10:25:40.187936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 06 10:25:40.187936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application yield resource Jun 06 10:25:40.187936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 10:25:40.187936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application next(self.gen) Jun 06 10:25:40.187936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 06 10:25:40.187936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self.session.rollback() Jun 06 10:25:40.187936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 06 10:25:40.187936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self.force_reraise() Jun 06 10:25:40.187936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 06 10:25:40.187936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise self.value Jun 06 10:25:40.187936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 06 10:25:40.187936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 06 10:25:40.187936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 06 10:25:40.187936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application session.commit() Jun 06 10:25:40.187936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 06 10:25:40.187936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 06 10:25:40.187936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 06 10:25:40.189440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._prepare_impl() Jun 06 10:25:40.189440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 06 10:25:40.189440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self.session.flush() Jun 06 10:25:40.189440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 06 10:25:40.189440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._flush(objects) Jun 06 10:25:40.189440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 06 10:25:40.189440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 06 10:25:40.189440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 06 10:25:40.189440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application compat.raise_( Jun 06 10:25:40.189440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 10:25:40.189440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception Jun 06 10:25:40.189440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 06 10:25:40.189440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application flush_context.execute() Jun 06 10:25:40.189440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 06 10:25:40.189440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rec.execute(self) Jun 06 10:25:40.189440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 06 10:25:40.189440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 06 10:25:40.192460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 06 10:25:40.192460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 06 10:25:40.192460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 06 10:25:40.192460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 06 10:25:40.192460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 06 10:25:40.192460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 06 10:25:40.192460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 06 10:25:40.192460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 06 10:25:40.192460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 06 10:25:40.192460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 06 10:25:40.192460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 06 10:25:40.192460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 06 10:25:40.192460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 06 10:25:40.192460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 06 10:25:40.192460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 10:25:40.192460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception Jun 06 10:25:40.193354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 10:25:40.193354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 10:25:40.193354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 10:25:40.193354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 10:25:40.193354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 10:25:40.193354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 10:25:40.193354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 10:25:40.193354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application conn.query(q) Jun 06 10:25:40.193354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 10:25:40.193354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 10:25:40.193354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 10:25:40.193354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application result.read() Jun 06 10:25:40.193354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 10:25:40.193354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 10:25:40.193354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 10:25:40.193354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 10:25:40.194212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 10:25:40.194212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 10:25:40.194212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 10:25:40.194212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 10:25:40.194212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 06 10:25:40.194212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 06 10:25:40.194212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [parameters: {'user_id': '048c6896f68a4d62befbb70e243ce40b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 06 10:25:40.194212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 06 10:25:40.194212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:40.194212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:40.194212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:40.194212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:40.194212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:40.194212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:40.194212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:40.195028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:40.195028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:40.195028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:40.195028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:40.195028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:40.195028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:40.195028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:40.195028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 06 10:25:40.195028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 06 10:25:40.195028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:40.195028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:40.195028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:40.195028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:40.195028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:40.195028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:40.195028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 06 10:25:40.195028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 06 10:25:40.196329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 06 10:25:40.196329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 06 10:25:40.196329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 06 10:25:40.196329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 06 10:25:40.196329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 06 10:25:40.196329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:40.196329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 485/963] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Jun 6 10:25:39 2022] POST /identity/v3/users => generated 160 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:40.198711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb30af-d2ae-4831-84d2-23765321dd95 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:40.237598 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:40.257084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d30d2-9041-4f2b-ad5b-faad27f06ee3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:40.300604 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1eb77-35a0-4b8a-b134-e12aba111bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:40.301648 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1eb77-35a0-4b8a-b134-e12aba111bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:40.302241 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1eb77-35a0-4b8a-b134-e12aba111bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:40.302718 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1eb77-35a0-4b8a-b134-e12aba111bd0 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:40.303664 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1eb77-35a0-4b8a-b134-e12aba111bd0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:40.304194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1eb77-35a0-4b8a-b134-e12aba111bd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:40.305796 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1eb77-35a0-4b8a-b134-e12aba111bd0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:40.311853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 479/964] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jun 6 10:25:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:40.317999 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1eb77-35a0-4b8a-b134-e12aba111bd0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:40.342178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb30af-d2ae-4831-84d2-23765321dd95 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:40.395704 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:40.397700 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d0c5c-dc43-4304-be1e-1b566a9c46f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:40.400031 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d0c5c-dc43-4304-be1e-1b566a9c46f8 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:40.400031 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d0c5c-dc43-4304-be1e-1b566a9c46f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:40.400031 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d0c5c-dc43-4304-be1e-1b566a9c46f8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:40.410341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934d0c5c-dc43-4304-be1e-1b566a9c46f8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:40.412231 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934d0c5c-dc43-4304-be1e-1b566a9c46f8 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:40.431757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-934d0c5c-dc43-4304-be1e-1b566a9c46f8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 10:25:40.431757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:40.431757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:40.431757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:40.431757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:40.431757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:40.431757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:40.431757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:40.431757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:40.431757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:40.431757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:40.431757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:40.431757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:25:40.431757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:25:40.431757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:25:40.431757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:25:40.432763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:40.432763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:40.432763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:40.432763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:40.432763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:40.432763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:40.432763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:40.432763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:40.432763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:40.432763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:40.432763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:40.432763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:40.432763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:40.432763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:40.432763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:40.432763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:40.432763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:40.432763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:40.433873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:40.433873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:40.433873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:25:40.433873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:25:40.433873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:25:40.433873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:25:40.433873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:25:40.433873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:25:40.433873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 10:25:40.433873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:40.438025 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 486/965] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:25:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:40.439517 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d0c5c-dc43-4304-be1e-1b566a9c46f8 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:40.446162 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1eb77-35a0-4b8a-b134-e12aba111bd0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:40.482078 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf0cff-5752-4787-8f46-8c205ce2defe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:40.483049 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf0cff-5752-4787-8f46-8c205ce2defe None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:40.483335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf0cff-5752-4787-8f46-8c205ce2defe None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:40.483613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf0cff-5752-4787-8f46-8c205ce2defe None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:40.484353 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf0cff-5752-4787-8f46-8c205ce2defe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:40.484662 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf0cff-5752-4787-8f46-8c205ce2defe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:40.486751 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf0cff-5752-4787-8f46-8c205ce2defe None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:40.492499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 480/966] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jun 6 10:25:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:40.497735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ceacf-eabd-4191-a677-b58c5c66ec11 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:40.498185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ceacf-eabd-4191-a677-b58c5c66ec11 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:40.498465 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ceacf-eabd-4191-a677-b58c5c66ec11 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:40.500295 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 481/967] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:40.504922 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ceacf-eabd-4191-a677-b58c5c66ec11 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:40.547524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:40.562781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d0c5c-dc43-4304-be1e-1b566a9c46f8 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:40.602118 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f00931-2055-4d23-ac8b-1a37d3e19646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:40.602797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f00931-2055-4d23-ac8b-1a37d3e19646 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:40.603081 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f00931-2055-4d23-ac8b-1a37d3e19646 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:40.603356 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f00931-2055-4d23-ac8b-1a37d3e19646 None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:40.604214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f00931-2055-4d23-ac8b-1a37d3e19646 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:40.604521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f00931-2055-4d23-ac8b-1a37d3e19646 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:40.605971 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f00931-2055-4d23-ac8b-1a37d3e19646 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:40.618044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ceacf-eabd-4191-a677-b58c5c66ec11 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:40.651161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:40.659877 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 487/968] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jun 6 10:25:40 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:40.659877 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52d274-4e7f-4692-afa9-cf2f11c5260b None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:40.659877 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52d274-4e7f-4692-afa9-cf2f11c5260b None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:40.659877 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52d274-4e7f-4692-afa9-cf2f11c5260b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:40.666418 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5e5cb-2adc-42be-a53e-90a59914ed6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:40.667709 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5e5cb-2adc-42be-a53e-90a59914ed6c None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:40.667709 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5e5cb-2adc-42be-a53e-90a59914ed6c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:40.667709 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5e5cb-2adc-42be-a53e-90a59914ed6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:40.668428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d5e5cb-2adc-42be-a53e-90a59914ed6c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:40.669837 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d5e5cb-2adc-42be-a53e-90a59914ed6c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:40.681728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-4a52d274-4e7f-4692-afa9-cf2f11c5260b None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:40.683505 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-4a52d274-4e7f-4692-afa9-cf2f11c5260b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:40.696687 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 482/969] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jun 6 10:25:40 2022] POST /identity/v3/endpoints => generated 349 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:25:40.795688 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 488/970] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:40 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:40.821496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5e5cb-2adc-42be-a53e-90a59914ed6c None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:40.865394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778270c1-521e-4ae3-b3b1-4fb419e9576b None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:40.865946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778270c1-521e-4ae3-b3b1-4fb419e9576b None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:40.865946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778270c1-521e-4ae3-b3b1-4fb419e9576b None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:40.867404 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 489/971] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:25:40.873578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f7e7c-8dfc-4360-9870-600f89e6f244 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:40.873878 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f7e7c-8dfc-4360-9870-600f89e6f244 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:40.874132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f7e7c-8dfc-4360-9870-600f89e6f244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:40.894785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-0e1f7e7c-8dfc-4360-9870-600f89e6f244 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:40.896043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-0e1f7e7c-8dfc-4360-9870-600f89e6f244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:40.951270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5e5cb-2adc-42be-a53e-90a59914ed6c None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:40.992841 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 490/972] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:40 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:40.996151 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9baa0d-92f0-4bef-bb30-c12726a0a90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:40.997184 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9baa0d-92f0-4bef-bb30-c12726a0a90d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:40.997467 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9baa0d-92f0-4bef-bb30-c12726a0a90d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:40.997738 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9baa0d-92f0-4bef-bb30-c12726a0a90d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:41.008675 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9baa0d-92f0-4bef-bb30-c12726a0a90d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:25:41.017702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f7e7c-8dfc-4360-9870-600f89e6f244 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:41.018433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-4f9baa0d-92f0-4bef-bb30-c12726a0a90d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 10:25:41.018433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:41.018433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:41.018433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:41.018433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:41.018433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:41.018433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:41.018433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:41.018433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:41.018433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:41.018433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:41.018433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:41.018433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:25:41.018433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:25:41.018433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:25:41.018433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:25:41.019312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:25:41.019312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:25:41.019312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:25:41.019312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:25:41.019312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:41.019312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:41.019312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:41.019312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:41.019312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:41.019312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:41.019312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:41.019312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:41.019312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:41.019312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:41.019312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:41.019312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:41.020289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:41.020289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:41.020289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:41.020289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:41.020289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:25:41.020289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:25:41.020289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:25:41.020289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:25:41.020289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:25:41.020289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:25:41.020289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 10:25:41.020289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:41.020289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 483/973] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Jun 6 10:25:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:41.021779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:41.029629 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9baa0d-92f0-4bef-bb30-c12726a0a90d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:41.080848 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:41.134823 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f7e7c-8dfc-4360-9870-600f89e6f244 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:41.138068 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9baa0d-92f0-4bef-bb30-c12726a0a90d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:41.174582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b6a84-d314-4c2e-8572-f76f488efbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:41.174818 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571c761-21cd-4a2c-913a-42fa34778b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:41.175322 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571c761-21cd-4a2c-913a-42fa34778b6b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:41.175609 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571c761-21cd-4a2c-913a-42fa34778b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:41.175916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571c761-21cd-4a2c-913a-42fa34778b6b None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:41.176568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b6a84-d314-4c2e-8572-f76f488efbc1 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:41.176743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571c761-21cd-4a2c-913a-42fa34778b6b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:41.177077 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571c761-21cd-4a2c-913a-42fa34778b6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:41.177498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b6a84-d314-4c2e-8572-f76f488efbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:41.177966 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b6a84-d314-4c2e-8572-f76f488efbc1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:41.178624 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571c761-21cd-4a2c-913a-42fa34778b6b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:41.179145 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b6a84-d314-4c2e-8572-f76f488efbc1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:41.180870 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b6a84-d314-4c2e-8572-f76f488efbc1 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:41.192760 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 484/974] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Jun 6 10:25:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:41.194341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-d98b6a84-d314-4c2e-8572-f76f488efbc1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 10:25:41.194341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:41.194341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:41.194341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:41.194341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:41.194341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:41.194341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:41.194341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:41.194341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 10:25:41.194341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:41.194341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:41.194341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:41.194341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:41.194341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:41.194341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:41.194341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:41.194341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:41.195355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:41.195355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:41.195355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:41.195355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:41.195355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:41.195355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:41.195355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:41.195355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:41.195355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:41.195355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:41.195355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:41.195355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:41.195355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:41.195355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:41.195355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:41.195355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:41.196260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:41.196260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:41.196260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:41.196260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:41.196260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:41.196260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:41.196260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:41.196260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:41.196260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:41.196260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:41.196260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:41.196260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:41.196260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 10:25:41.196260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:41.196260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 491/975] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:41.201663 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571c761-21cd-4a2c-913a-42fa34778b6b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:41.202418 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b6a84-d314-4c2e-8572-f76f488efbc1 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:41.272380 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:41.314639 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b6a84-d314-4c2e-8572-f76f488efbc1 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:41.321129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571c761-21cd-4a2c-913a-42fa34778b6b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:41.328949 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:41.348730 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014b345-00f8-4ba6-9ee9-12d1b146f7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:41.349749 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014b345-00f8-4ba6-9ee9-12d1b146f7a6 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:41.350192 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014b345-00f8-4ba6-9ee9-12d1b146f7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:41.350467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014b345-00f8-4ba6-9ee9-12d1b146f7a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:41.358575 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae7044-1db6-4eb9-80ad-3bac1483d2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:41.359357 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae7044-1db6-4eb9-80ad-3bac1483d2dd None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:41.359638 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae7044-1db6-4eb9-80ad-3bac1483d2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:41.359946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae7044-1db6-4eb9-80ad-3bac1483d2dd None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:41.361192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae7044-1db6-4eb9-80ad-3bac1483d2dd None admin] RBAC: Adding query filter params (name=compute) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:41.361498 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae7044-1db6-4eb9-80ad-3bac1483d2dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:41.361745 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014b345-00f8-4ba6-9ee9-12d1b146f7a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:41.362860 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae7044-1db6-4eb9-80ad-3bac1483d2dd None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:41.363142 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014b345-00f8-4ba6-9ee9-12d1b146f7a6 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:41.372972 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 485/976] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jun 6 10:25:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:41.373756 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae7044-1db6-4eb9-80ad-3bac1483d2dd None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:41.387711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-2014b345-00f8-4ba6-9ee9-12d1b146f7a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:41.387711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:41.387711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:41.387711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:41.387711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:41.387711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:41.387711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:41.387711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:41.387711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:41.387711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:41.387711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:41.387711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:41.387711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:41.387711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:41.387711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:41.388818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:41.388818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:41.388818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:41.388818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:41.388818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:41.388818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:41.388818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:41.388818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:41.388818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:41.388818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:41.388818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:41.388818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:41.388818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:41.388818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:41.388818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:41.388818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:41.388818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:41.389779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:41.389779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:41.389779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:41.389779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:41.389779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 492/977] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:41.395737 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014b345-00f8-4ba6-9ee9-12d1b146f7a6 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:41.484779 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae7044-1db6-4eb9-80ad-3bac1483d2dd None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:41.504635 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014b345-00f8-4ba6-9ee9-12d1b146f7a6 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:41.519494 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76ccc9-6aa6-4caf-8c39-7ca4e8a16383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:41.520192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76ccc9-6aa6-4caf-8c39-7ca4e8a16383 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:41.520474 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76ccc9-6aa6-4caf-8c39-7ca4e8a16383 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:41.520749 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76ccc9-6aa6-4caf-8c39-7ca4e8a16383 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:41.521451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76ccc9-6aa6-4caf-8c39-7ca4e8a16383 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:41.521754 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76ccc9-6aa6-4caf-8c39-7ca4e8a16383 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:41.523382 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76ccc9-6aa6-4caf-8c39-7ca4e8a16383 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:41.523807 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:41.528843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 486/978] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jun 6 10:25:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:41.534388 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76ccc9-6aa6-4caf-8c39-7ca4e8a16383 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:41.543407 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6d3ae-f393-4b60-8c31-979d5107708d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:41.544101 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6d3ae-f393-4b60-8c31-979d5107708d None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:41.544386 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6d3ae-f393-4b60-8c31-979d5107708d None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:41.544662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6d3ae-f393-4b60-8c31-979d5107708d None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:41.545379 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6d3ae-f393-4b60-8c31-979d5107708d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:41.545690 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6d3ae-f393-4b60-8c31-979d5107708d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:41.547064 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6d3ae-f393-4b60-8c31-979d5107708d None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:41.567298 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 493/979] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:41.568725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6d3ae-f393-4b60-8c31-979d5107708d None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:41.570362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:41.639023 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76ccc9-6aa6-4caf-8c39-7ca4e8a16383 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:41.673686 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc38c74-92f2-4816-ab56-00c43311020e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:41.674602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc38c74-92f2-4816-ab56-00c43311020e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:41.675038 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc38c74-92f2-4816-ab56-00c43311020e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:41.675453 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc38c74-92f2-4816-ab56-00c43311020e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:41.676440 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc38c74-92f2-4816-ab56-00c43311020e None admin] RBAC: Adding query filter params (interface=public, service_id=8b6286170ec24c6680840356863e5359, region_id=RegionOne) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:41.676898 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc38c74-92f2-4816-ab56-00c43311020e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:41.677990 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6d3ae-f393-4b60-8c31-979d5107708d None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:41.678627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc38c74-92f2-4816-ab56-00c43311020e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:41.684836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 487/980] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jun 6 10:25:41 2022] GET /identity/v3/endpoints?service_id=8b6286170ec24c6680840356863e5359&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:41.690715 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc38c74-92f2-4816-ab56-00c43311020e None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:41.713953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3ae07-5a9c-4da9-8ada-ad75f8f77e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:41.714749 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3ae07-5a9c-4da9-8ada-ad75f8f77e23 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:41.715029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3ae07-5a9c-4da9-8ada-ad75f8f77e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:41.715301 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3ae07-5a9c-4da9-8ada-ad75f8f77e23 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:41.722932 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3ae07-5a9c-4da9-8ada-ad75f8f77e23 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:41.724385 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3ae07-5a9c-4da9-8ada-ad75f8f77e23 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:41.739462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-6de3ae07-5a9c-4da9-8ada-ad75f8f77e23 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 10:25:41.739462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:41.739462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:41.739462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:41.739462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:41.739462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:41.739462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:41.739462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:41.739462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:41.739462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:41.739462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:41.739462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:41.739462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:25:41.739462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:25:41.739462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:25:41.739462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:25:41.740648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:41.740648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:41.740648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:41.740648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:41.740648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:41.740648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:41.740648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:41.740648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:41.740648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:41.740648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:41.740648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:41.740648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:41.740648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:41.740648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:41.740648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:41.740648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:41.740648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:41.740648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:41.741810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:41.741810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:41.741810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:25:41.741810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:25:41.741810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:25:41.741810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:25:41.741810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:25:41.741810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:25:41.741810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 10:25:41.741810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:41.745400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 494/981] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jun 6 10:25:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:41.746223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3ae07-5a9c-4da9-8ada-ad75f8f77e23 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:41.787502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:41.793976 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:41.798933 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc38c74-92f2-4816-ab56-00c43311020e None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:41.835404 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89eb143-8785-4048-9771-e67dc640db22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:41.836762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89eb143-8785-4048-9771-e67dc640db22 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:41.837197 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89eb143-8785-4048-9771-e67dc640db22 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:41.838073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89eb143-8785-4048-9771-e67dc640db22 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:41.839203 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89eb143-8785-4048-9771-e67dc640db22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:41.841230 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89eb143-8785-4048-9771-e67dc640db22 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:41.850102 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3ae07-5a9c-4da9-8ada-ad75f8f77e23 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:41.850693 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 488/982] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jun 6 10:25:41 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:25:41.891118 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755417e-8861-4013-86be-dac475b78d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:41.892402 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755417e-8861-4013-86be-dac475b78d54 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:41.892650 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755417e-8861-4013-86be-dac475b78d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:41.892924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755417e-8861-4013-86be-dac475b78d54 None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:41.893780 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755417e-8861-4013-86be-dac475b78d54 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:41.894091 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755417e-8861-4013-86be-dac475b78d54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:41.895550 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755417e-8861-4013-86be-dac475b78d54 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:41.917020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 495/983] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Jun 6 10:25:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:41.923431 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89eb143-8785-4048-9771-e67dc640db22 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:42.033752 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89eb143-8785-4048-9771-e67dc640db22 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:42.072012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40345505-3423-4efb-9020-177c4611e51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:42.073050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40345505-3423-4efb-9020-177c4611e51c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:42.073534 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40345505-3423-4efb-9020-177c4611e51c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:42.074018 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40345505-3423-4efb-9020-177c4611e51c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:42.106979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:42.147351 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40345505-3423-4efb-9020-177c4611e51c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:42.149063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40345505-3423-4efb-9020-177c4611e51c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:42.253882 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 489/984] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:25:41 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 10:25:42.255095 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755417e-8861-4013-86be-dac475b78d54 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:42.349297 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:42.356765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755417e-8861-4013-86be-dac475b78d54 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:42.399714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ae5a8-c074-4121-bee2-701a06b4abb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:42.400534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ae5a8-c074-4121-bee2-701a06b4abb0 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:42.400816 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ae5a8-c074-4121-bee2-701a06b4abb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:42.401088 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ae5a8-c074-4121-bee2-701a06b4abb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:42.411959 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ae5a8-c074-4121-bee2-701a06b4abb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:42.413397 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ae5a8-c074-4121-bee2-701a06b4abb0 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:42.431270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-fb3ae5a8-c074-4121-bee2-701a06b4abb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:42.431270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:42.431270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:42.431270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:42.431270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:42.431270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:42.431270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:42.431270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:42.431270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:42.431270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:42.431270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:42.431270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:42.431270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:42.431270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:42.431270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:42.432394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:42.432394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:42.432394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:42.432394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:42.432394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:42.432394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:42.432394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:42.432394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:42.432394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:42.432394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:42.432394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:42.432394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:42.432394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:42.432394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:42.432394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:42.432394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:42.432394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:42.438052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:42.438052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:42.438052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:42.438052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:42.438052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 496/985] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:42.438345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40345505-3423-4efb-9020-177c4611e51c None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:42.500020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:42.540742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40345505-3423-4efb-9020-177c4611e51c None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:42.574975 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd68545-3ff4-4370-b9b3-bc09aeaab5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:42.575643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd68545-3ff4-4370-b9b3-bc09aeaab5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:42.575954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd68545-3ff4-4370-b9b3-bc09aeaab5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:42.576228 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd68545-3ff4-4370-b9b3-bc09aeaab5e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:42.576938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd68545-3ff4-4370-b9b3-bc09aeaab5e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:42.577242 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd68545-3ff4-4370-b9b3-bc09aeaab5e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:42.578599 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd68545-3ff4-4370-b9b3-bc09aeaab5e5 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:42.595598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 490/986] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:42.597009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ae5a8-c074-4121-bee2-701a06b4abb0 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:42.698153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ae5a8-c074-4121-bee2-701a06b4abb0 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:42.735091 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22404c46-ff42-4ca6-9ee2-b3a128e7007e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:42.736246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22404c46-ff42-4ca6-9ee2-b3a128e7007e None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:42.736534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22404c46-ff42-4ca6-9ee2-b3a128e7007e None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:42.736809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22404c46-ff42-4ca6-9ee2-b3a128e7007e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:42.744652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22404c46-ff42-4ca6-9ee2-b3a128e7007e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:42.746144 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22404c46-ff42-4ca6-9ee2-b3a128e7007e None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:42.761075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-22404c46-ff42-4ca6-9ee2-b3a128e7007e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:42.761075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:42.761075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:42.761075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:42.761075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:42.761075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:42.761075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:42.761075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:42.761075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:42.761075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:42.761075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:42.761075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:42.761075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:42.761075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:42.761075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:42.762112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:42.762112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:42.762112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:42.762112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:42.762112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:42.762112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:42.762112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:42.762112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:42.762112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:42.762112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:42.762112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:42.762112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:42.762112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:42.762112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:42.762112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:42.762112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:42.762112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:42.768438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:42.768438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:42.768438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:42.768438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:42.768438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:42.768438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:42.768438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:42.768438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:42.768438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 497/987] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:42.768926 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd68545-3ff4-4370-b9b3-bc09aeaab5e5 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:42.869320 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd68545-3ff4-4370-b9b3-bc09aeaab5e5 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:42.893042 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:42.905323 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269b4c7-a162-4fc8-9fce-451e6ff55f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:42.905981 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269b4c7-a162-4fc8-9fce-451e6ff55f26 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:42.906260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269b4c7-a162-4fc8-9fce-451e6ff55f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:42.906532 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269b4c7-a162-4fc8-9fce-451e6ff55f26 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:42.907351 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269b4c7-a162-4fc8-9fce-451e6ff55f26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:42.907932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269b4c7-a162-4fc8-9fce-451e6ff55f26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:42.909465 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269b4c7-a162-4fc8-9fce-451e6ff55f26 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:42.921492 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 491/988] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:25:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:42.928160 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22404c46-ff42-4ca6-9ee2-b3a128e7007e None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:42.941543 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:43.036367 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22404c46-ff42-4ca6-9ee2-b3a128e7007e None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:43.063964 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002497e7-7b76-45b4-8966-800927afa40e None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:43.064263 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002497e7-7b76-45b4-8966-800927afa40e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:43.064520 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002497e7-7b76-45b4-8966-800927afa40e None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:43.066062 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 492/989] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:43.070850 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d24d7b-5068-442a-b0b3-cfe979c19236 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:43.071147 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d24d7b-5068-442a-b0b3-cfe979c19236 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:43.071249 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a945971-e040-4bfa-b0b6-44e9c1ecee5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:43.071439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d24d7b-5068-442a-b0b3-cfe979c19236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:43.071990 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a945971-e040-4bfa-b0b6-44e9c1ecee5c None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:43.072272 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a945971-e040-4bfa-b0b6-44e9c1ecee5c None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:43.072544 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a945971-e040-4bfa-b0b6-44e9c1ecee5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:43.073367 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a945971-e040-4bfa-b0b6-44e9c1ecee5c None admin] RBAC: Adding query filter params (role.id=68d664d01d3340a58359d3fc9d738142, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=e9e358afe0974c0c8bd53104ebbff445) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:43.073681 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a945971-e040-4bfa-b0b6-44e9c1ecee5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:43.075097 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a945971-e040-4bfa-b0b6-44e9c1ecee5c None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:43.081894 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 498/990] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:25:42 2022] GET /identity/v3/role_assignments?user.id=e9e358afe0974c0c8bd53104ebbff445&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=68d664d01d3340a58359d3fc9d738142 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:43.095342 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-e9d24d7b-5068-442a-b0b3-cfe979c19236 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:43.096729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e9d24d7b-5068-442a-b0b3-cfe979c19236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:43.172465 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 493/991] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:43 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 10:25:43.196879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a945971-e040-4bfa-b0b6-44e9c1ecee5c None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:43.274508 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:43.308616 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a945971-e040-4bfa-b0b6-44e9c1ecee5c None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:43.345267 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1935a7a-9f71-4456-8567-26bc970f6f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:43.346079 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1935a7a-9f71-4456-8567-26bc970f6f8c None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:43.346358 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1935a7a-9f71-4456-8567-26bc970f6f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:43.346628 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1935a7a-9f71-4456-8567-26bc970f6f8c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:43.347274 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1935a7a-9f71-4456-8567-26bc970f6f8c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:43.348651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1935a7a-9f71-4456-8567-26bc970f6f8c None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:43.361389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-b1935a7a-9f71-4456-8567-26bc970f6f8c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 10:25:43.361389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:43.361389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:43.361389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:43.361389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:43.361389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:43.361389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:43.361389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:43.361389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 10:25:43.361389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:43.361389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:43.361389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:43.361389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:43.361389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:43.361389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:43.361389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:43.361389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:43.362389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:43.362389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:43.362389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:43.362389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:43.362389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:43.362389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:43.362389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:43.362389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:43.362389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:43.362389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:43.362389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:43.362389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:43.362389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:43.362389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:43.362389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:43.362389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:43.363632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:43.363632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:43.363632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:43.363632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:43.363632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:43.363632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:43.363632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:43.363632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:43.363632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:43.363632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:43.363632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:43.363632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:43.363632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 10:25:43.363632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:43.363632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 499/992] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jun 6 10:25:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:43.368822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d24d7b-5068-442a-b0b3-cfe979c19236 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:43.472462 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d24d7b-5068-442a-b0b3-cfe979c19236 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:43.486554 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:43.508401 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622d5c5-133a-4983-9a01-8a210bae164f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:43.509069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622d5c5-133a-4983-9a01-8a210bae164f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:43.509354 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622d5c5-133a-4983-9a01-8a210bae164f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:43.509677 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622d5c5-133a-4983-9a01-8a210bae164f None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:43.510380 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622d5c5-133a-4983-9a01-8a210bae164f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:43.510732 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622d5c5-133a-4983-9a01-8a210bae164f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:43.512118 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622d5c5-133a-4983-9a01-8a210bae164f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:43.517754 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 494/993] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jun 6 10:25:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:43.523422 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1935a7a-9f71-4456-8567-26bc970f6f8c None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:43.624327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1935a7a-9f71-4456-8567-26bc970f6f8c None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:43.659666 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868064ef-97e1-4f95-b8e8-dd7a58cbe69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:43.660356 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868064ef-97e1-4f95-b8e8-dd7a58cbe69d None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:43.660637 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868064ef-97e1-4f95-b8e8-dd7a58cbe69d None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:43.660909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868064ef-97e1-4f95-b8e8-dd7a58cbe69d None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:43.661609 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868064ef-97e1-4f95-b8e8-dd7a58cbe69d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:43.661910 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868064ef-97e1-4f95-b8e8-dd7a58cbe69d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:43.663346 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868064ef-97e1-4f95-b8e8-dd7a58cbe69d None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:43.665643 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:43.675198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 500/994] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jun 6 10:25:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:43.676555 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622d5c5-133a-4983-9a01-8a210bae164f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:43.780931 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622d5c5-133a-4983-9a01-8a210bae164f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:43.817213 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bc8fd-6263-4226-9378-3d9d468d39b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:43.817907 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bc8fd-6263-4226-9378-3d9d468d39b0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:43.818189 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bc8fd-6263-4226-9378-3d9d468d39b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:43.818462 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bc8fd-6263-4226-9378-3d9d468d39b0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:43.819287 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665bc8fd-6263-4226-9378-3d9d468d39b0 None admin] RBAC: Adding query filter params (interface=admin, service_id=cc2623eebf484287b63e512e59444110, region_id=RegionOne) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:43.819597 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665bc8fd-6263-4226-9378-3d9d468d39b0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:43.821050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665bc8fd-6263-4226-9378-3d9d468d39b0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:43.831684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 495/995] 10.222.0.42 () {56 vars in 1249 bytes} [Mon Jun 6 10:25:43 2022] GET /identity/v3/endpoints?service_id=cc2623eebf484287b63e512e59444110&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:43.833193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868064ef-97e1-4f95-b8e8-dd7a58cbe69d None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:43.936333 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868064ef-97e1-4f95-b8e8-dd7a58cbe69d None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:43.969005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a7f24-a4b0-4486-a582-8df3f9a83ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:43.970262 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a7f24-a4b0-4486-a582-8df3f9a83ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:43.970714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a7f24-a4b0-4486-a582-8df3f9a83ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:43.970998 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a7f24-a4b0-4486-a582-8df3f9a83ca9 None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:43.971668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a7f24-a4b0-4486-a582-8df3f9a83ca9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:43.973060 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a7f24-a4b0-4486-a582-8df3f9a83ca9 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:43.979591 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 501/996] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jun 6 10:25:43 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:25:44.163307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0eb8b-0a30-43ad-af04-338636ad2569 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:44.163843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0eb8b-0a30-43ad-af04-338636ad2569 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:44.163952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0eb8b-0a30-43ad-af04-338636ad2569 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:44.165542 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 496/997] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:44.170946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1a966-2b06-4f48-9b81-ad8dc26994f1 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:44.171419 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1a966-2b06-4f48-9b81-ad8dc26994f1 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:44.171699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1a966-2b06-4f48-9b81-ad8dc26994f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:44.189874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-26b1a966-2b06-4f48-9b81-ad8dc26994f1 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:44.190998 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-26b1a966-2b06-4f48-9b81-ad8dc26994f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:44.252464 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:44.262750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 502/998] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:44 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:44.287493 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0eb8b-0a30-43ad-af04-338636ad2569 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:44.332406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:44.404911 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0eb8b-0a30-43ad-af04-338636ad2569 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:44.440663 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3868f-8e12-4ff4-9d9a-465e811ae581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:44.441381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3868f-8e12-4ff4-9d9a-465e811ae581 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:44.441663 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3868f-8e12-4ff4-9d9a-465e811ae581 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:44.441938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3868f-8e12-4ff4-9d9a-465e811ae581 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:44.442584 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed3868f-8e12-4ff4-9d9a-465e811ae581 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:44.443979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed3868f-8e12-4ff4-9d9a-465e811ae581 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:44.456615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-0ed3868f-8e12-4ff4-9d9a-465e811ae581 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 10:25:44.456615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:44.456615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:44.456615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:44.456615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:44.456615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:44.456615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:44.456615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:44.456615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 10:25:44.456615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:44.456615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:44.456615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:44.456615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:44.456615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:44.456615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:44.456615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:44.456615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:44.457593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:44.457593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:44.457593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:44.457593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:44.457593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:44.457593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:44.457593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:44.457593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:44.457593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:44.457593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:44.457593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:44.457593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:44.457593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:44.457593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:44.457593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:44.457593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:44.458439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:44.458439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:44.458439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:44.458439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:44.458439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:44.458439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:44.458439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:44.458439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:44.458439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:44.458439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:44.458439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:44.458439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:44.458439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 10:25:44.458439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:44.459972 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 497/999] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:44.463877 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1a966-2b06-4f48-9b81-ad8dc26994f1 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:44.566570 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1a966-2b06-4f48-9b81-ad8dc26994f1 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:44.604693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff68e02-4a84-42da-83c2-8b891bb956d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:44.605355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff68e02-4a84-42da-83c2-8b891bb956d0 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:44.605634 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff68e02-4a84-42da-83c2-8b891bb956d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:44.605906 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff68e02-4a84-42da-83c2-8b891bb956d0 None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:44.606605 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff68e02-4a84-42da-83c2-8b891bb956d0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:44.606906 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff68e02-4a84-42da-83c2-8b891bb956d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:44.608306 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff68e02-4a84-42da-83c2-8b891bb956d0 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:44.617893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 503/1000] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jun 6 10:25:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:44.619000 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3868f-8e12-4ff4-9d9a-465e811ae581 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:44.728152 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3868f-8e12-4ff4-9d9a-465e811ae581 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:44.731248 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:44.763927 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a93b2f-9653-4b6d-89e3-aeda2456ae6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:44.764601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a93b2f-9653-4b6d-89e3-aeda2456ae6c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:44.764887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a93b2f-9653-4b6d-89e3-aeda2456ae6c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:44.765165 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a93b2f-9653-4b6d-89e3-aeda2456ae6c None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:44.765876 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a93b2f-9653-4b6d-89e3-aeda2456ae6c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:44.766186 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a93b2f-9653-4b6d-89e3-aeda2456ae6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:44.768379 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a93b2f-9653-4b6d-89e3-aeda2456ae6c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:44.778394 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 498/1001] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jun 6 10:25:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:44.779489 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff68e02-4a84-42da-83c2-8b891bb956d0 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:44.840378 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:44.884184 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff68e02-4a84-42da-83c2-8b891bb956d0 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:44.918114 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea69e7-dff9-4f84-9455-68abe3d6fa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:44.918771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea69e7-dff9-4f84-9455-68abe3d6fa15 None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:44.919056 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea69e7-dff9-4f84-9455-68abe3d6fa15 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:44.919331 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea69e7-dff9-4f84-9455-68abe3d6fa15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:44.920104 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea69e7-dff9-4f84-9455-68abe3d6fa15 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:44.921460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea69e7-dff9-4f84-9455-68abe3d6fa15 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:44.942044 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 504/1002] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jun 6 10:25:44 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:25:45.423979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9ba62-c1af-4b1b-b719-a721eb1d008d None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:45.424528 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9ba62-c1af-4b1b-b719-a721eb1d008d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:45.424634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9ba62-c1af-4b1b-b719-a721eb1d008d None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:45.426126 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 499/1003] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:45.431232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd20ce-5d51-43a5-ab95-e2c74ed3a5d2 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:45.431522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd20ce-5d51-43a5-ab95-e2c74ed3a5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:45.431812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd20ce-5d51-43a5-ab95-e2c74ed3a5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:45.449508 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-a3cd20ce-5d51-43a5-ab95-e2c74ed3a5d2 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:45.450690 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a3cd20ce-5d51-43a5-ab95-e2c74ed3a5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:45.525558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 505/1004] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:45 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:45.547673 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9ba62-c1af-4b1b-b719-a721eb1d008d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:45.652885 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9ba62-c1af-4b1b-b719-a721eb1d008d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:45.686153 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045340b6-b464-42a8-9d6b-284188378b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:45.686951 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045340b6-b464-42a8-9d6b-284188378b36 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:45.687236 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045340b6-b464-42a8-9d6b-284188378b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:45.687509 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045340b6-b464-42a8-9d6b-284188378b36 None admin] PATH_INFO: `/v3/services/image` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:45.688203 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045340b6-b464-42a8-9d6b-284188378b36 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:45.690194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045340b6-b464-42a8-9d6b-284188378b36 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:45.701844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-045340b6-b464-42a8-9d6b-284188378b36 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 10:25:45.701844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:45.701844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:45.701844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:45.701844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:45.701844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:45.701844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:45.701844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:45.701844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 10:25:45.701844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:45.701844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:45.701844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:45.701844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:45.701844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:45.701844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:45.701844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:45.701844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:45.702859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:45.702859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:45.702859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:45.702859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:45.702859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:45.702859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:45.702859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:45.702859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:45.702859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:45.702859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:45.702859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:45.702859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:45.702859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:45.702859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:45.702859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:45.702859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:45.703707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:45.703707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:45.703707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:45.703707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:45.703707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:45.703707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:45.703707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:45.703707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:45.703707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:45.703707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:45.703707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:45.703707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:45.703707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 10:25:45.703707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:45.708129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 500/1005] 10.222.0.42 () {56 vars in 1105 bytes} [Mon Jun 6 10:25:45 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:45.709859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd20ce-5d51-43a5-ab95-e2c74ed3a5d2 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:45.752436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:45.813468 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd20ce-5d51-43a5-ab95-e2c74ed3a5d2 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:45.848325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407f116-1db7-4f0e-945f-17138e928a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:45.848992 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407f116-1db7-4f0e-945f-17138e928a6f None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:45.849277 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407f116-1db7-4f0e-945f-17138e928a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:45.849555 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407f116-1db7-4f0e-945f-17138e928a6f None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:45.850257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407f116-1db7-4f0e-945f-17138e928a6f None admin] RBAC: Adding query filter params (name=image) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:45.850565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407f116-1db7-4f0e-945f-17138e928a6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:45.851956 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407f116-1db7-4f0e-945f-17138e928a6f None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:45.861638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 506/1006] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:45.862850 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045340b6-b464-42a8-9d6b-284188378b36 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:45.896794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:45.970979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045340b6-b464-42a8-9d6b-284188378b36 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:46.008117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94536069-1935-416b-9027-e7101f5f18fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:46.008798 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94536069-1935-416b-9027-e7101f5f18fa None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:46.009078 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94536069-1935-416b-9027-e7101f5f18fa None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:46.009350 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94536069-1935-416b-9027-e7101f5f18fa None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:46.010067 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94536069-1935-416b-9027-e7101f5f18fa None admin] RBAC: Adding query filter params (type=image) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:46.010368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94536069-1935-416b-9027-e7101f5f18fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:46.011753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94536069-1935-416b-9027-e7101f5f18fa None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:46.018442 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 501/1007] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:45 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:46.328022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42528d-7189-46db-960f-e42e25f2996c None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:46.328533 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42528d-7189-46db-960f-e42e25f2996c None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:46.328533 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42528d-7189-46db-960f-e42e25f2996c None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:46.330095 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 507/1008] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:46.335565 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680c8d1-2e47-49be-85b2-e5a852b34b6a None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:46.335950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680c8d1-2e47-49be-85b2-e5a852b34b6a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:46.336295 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680c8d1-2e47-49be-85b2-e5a852b34b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:46.353631 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9680c8d1-2e47-49be-85b2-e5a852b34b6a None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:46.354845 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9680c8d1-2e47-49be-85b2-e5a852b34b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:46.430721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 502/1009] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:46 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:46.453173 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42528d-7189-46db-960f-e42e25f2996c None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:46.557683 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42528d-7189-46db-960f-e42e25f2996c None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:46.560035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:46.593725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c655ef5-81e0-4833-b4a0-7e20b2da0d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:46.594492 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c655ef5-81e0-4833-b4a0-7e20b2da0d16 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:46.594810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c655ef5-81e0-4833-b4a0-7e20b2da0d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:46.595169 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c655ef5-81e0-4833-b4a0-7e20b2da0d16 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:46.595851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c655ef5-81e0-4833-b4a0-7e20b2da0d16 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:46.597211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c655ef5-81e0-4833-b4a0-7e20b2da0d16 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:46.609129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-9c655ef5-81e0-4833-b4a0-7e20b2da0d16 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 10:25:46.609129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:46.609129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:46.609129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:46.609129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:46.609129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:46.609129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:46.609129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:46.609129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 10:25:46.609129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:46.609129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:46.609129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:46.609129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:46.609129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:46.609129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:46.609129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:46.609129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:46.610630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:46.610630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:46.610630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:46.610630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:46.610630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:46.610630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:46.610630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:46.610630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:46.610630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:46.610630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:46.610630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:46.610630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:46.610630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:46.610630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:46.610630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:46.610630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:46.611652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:46.611652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:46.611652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:46.611652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:46.611652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:46.611652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:46.611652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:46.611652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:46.611652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:46.611652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:46.611652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:46.611652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:46.611652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 10:25:46.611652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:46.616047 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 508/1010] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jun 6 10:25:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:46.617263 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680c8d1-2e47-49be-85b2-e5a852b34b6a None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:46.644885 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:46.722016 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680c8d1-2e47-49be-85b2-e5a852b34b6a None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:46.756518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706e772-6388-4873-ace7-436c03b0c4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:46.757202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706e772-6388-4873-ace7-436c03b0c4bc None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:46.757485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706e772-6388-4873-ace7-436c03b0c4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:46.757761 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706e772-6388-4873-ace7-436c03b0c4bc None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:46.758474 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8706e772-6388-4873-ace7-436c03b0c4bc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:46.758780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8706e772-6388-4873-ace7-436c03b0c4bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:46.760182 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8706e772-6388-4873-ace7-436c03b0c4bc None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:46.770979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 503/1011] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jun 6 10:25:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:46.772150 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c655ef5-81e0-4833-b4a0-7e20b2da0d16 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:46.872590 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c655ef5-81e0-4833-b4a0-7e20b2da0d16 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:46.907345 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cfb71a-ca4c-4dc3-b85c-49d88f6a6052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:46.908034 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cfb71a-ca4c-4dc3-b85c-49d88f6a6052 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:46.908320 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cfb71a-ca4c-4dc3-b85c-49d88f6a6052 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:46.908599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cfb71a-ca4c-4dc3-b85c-49d88f6a6052 None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:46.909298 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cfb71a-ca4c-4dc3-b85c-49d88f6a6052 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:46.909606 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cfb71a-ca4c-4dc3-b85c-49d88f6a6052 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:46.910960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cfb71a-ca4c-4dc3-b85c-49d88f6a6052 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:46.920458 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 509/1012] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jun 6 10:25:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:46.921459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706e772-6388-4873-ace7-436c03b0c4bc None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:47.017506 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:47.028414 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706e772-6388-4873-ace7-436c03b0c4bc None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:47.061970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2244ecf-87eb-48ec-8d1b-3cf384d97d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:47.062631 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2244ecf-87eb-48ec-8d1b-3cf384d97d38 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:47.062917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2244ecf-87eb-48ec-8d1b-3cf384d97d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:47.063717 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2244ecf-87eb-48ec-8d1b-3cf384d97d38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:47.065092 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2244ecf-87eb-48ec-8d1b-3cf384d97d38 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:47.066754 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2244ecf-87eb-48ec-8d1b-3cf384d97d38 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:47.087818 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 504/1013] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jun 6 10:25:46 2022] POST /identity/v3/endpoints => generated 325 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:25:47.324430 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6507fc5-dffa-4e5e-97c5-baee205cfd93 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:47.324806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6507fc5-dffa-4e5e-97c5-baee205cfd93 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:47.325019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6507fc5-dffa-4e5e-97c5-baee205cfd93 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:47.326523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 510/1014] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:47.331828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15136e1a-17b7-42d3-97c8-a8ccadc96448 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:47.332129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15136e1a-17b7-42d3-97c8-a8ccadc96448 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:47.332410 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15136e1a-17b7-42d3-97c8-a8ccadc96448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:47.349006 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-15136e1a-17b7-42d3-97c8-a8ccadc96448 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:47.350184 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-15136e1a-17b7-42d3-97c8-a8ccadc96448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:47.428077 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 505/1015] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:47 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:47.452324 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6507fc5-dffa-4e5e-97c5-baee205cfd93 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:47.487279 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:47.561199 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6507fc5-dffa-4e5e-97c5-baee205cfd93 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:47.595248 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e7a4d-b3c2-42e6-bc60-1632ef8ba582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:47.595970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e7a4d-b3c2-42e6-bc60-1632ef8ba582 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:47.596249 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e7a4d-b3c2-42e6-bc60-1632ef8ba582 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:47.596516 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e7a4d-b3c2-42e6-bc60-1632ef8ba582 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:47.608468 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e7a4d-b3c2-42e6-bc60-1632ef8ba582 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:25:47.617561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-567e7a4d-b3c2-42e6-bc60-1632ef8ba582 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 10:25:47.617561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:47.617561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:47.617561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:47.617561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:47.617561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:47.617561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:47.617561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:47.617561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:47.617561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:47.617561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:47.617561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:47.617561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:25:47.617561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:25:47.617561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:25:47.617561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:25:47.618480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:25:47.618480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:25:47.618480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:25:47.618480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:25:47.618480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:47.618480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:47.618480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:47.618480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:47.618480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:47.618480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:47.618480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:47.618480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:47.618480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:47.618480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:47.618480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:47.618480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:47.619337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:47.619337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:47.619337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:47.619337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:47.619337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:25:47.619337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:25:47.619337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:25:47.619337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:25:47.619337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:25:47.619337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:25:47.619337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 10:25:47.619337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:47.623810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 511/1016] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Jun 6 10:25:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:47.625020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15136e1a-17b7-42d3-97c8-a8ccadc96448 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:47.703605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:47.729227 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15136e1a-17b7-42d3-97c8-a8ccadc96448 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:47.785375 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651bbe61-5657-40f9-bd71-ff2ddf53865c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:47.786050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bbe61-5657-40f9-bd71-ff2ddf53865c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:47.786326 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bbe61-5657-40f9-bd71-ff2ddf53865c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:47.786594 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bbe61-5657-40f9-bd71-ff2ddf53865c None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:47.787337 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651bbe61-5657-40f9-bd71-ff2ddf53865c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:47.787639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651bbe61-5657-40f9-bd71-ff2ddf53865c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:47.789037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651bbe61-5657-40f9-bd71-ff2ddf53865c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:47.804139 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 506/1017] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Jun 6 10:25:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:47.809803 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e7a4d-b3c2-42e6-bc60-1632ef8ba582 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:47.914238 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:47.924820 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e7a4d-b3c2-42e6-bc60-1632ef8ba582 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:47.962272 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae96de-6fef-424b-89fd-7ee91d680049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:47.963083 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae96de-6fef-424b-89fd-7ee91d680049 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:47.963361 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae96de-6fef-424b-89fd-7ee91d680049 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:47.963631 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae96de-6fef-424b-89fd-7ee91d680049 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:47.976419 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae96de-6fef-424b-89fd-7ee91d680049 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:47.977894 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae96de-6fef-424b-89fd-7ee91d680049 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:47.997768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-78ae96de-6fef-424b-89fd-7ee91d680049 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:47.997768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:47.997768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:47.997768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:47.997768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:47.997768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:47.997768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:47.997768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:47.997768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:47.997768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:47.997768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:47.997768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:47.997768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:47.997768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:47.997768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:47.998812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:47.998812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:47.998812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:47.998812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:47.998812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:47.998812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:47.998812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:47.998812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:47.998812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:47.998812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:47.998812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:47.998812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:47.998812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:47.998812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:47.998812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:47.998812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:47.998812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:47.999694 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:47.999694 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:47.999694 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:47.999694 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:48.005644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651bbe61-5657-40f9-bd71-ff2ddf53865c None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:48.006090 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 512/1018] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:48.108378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651bbe61-5657-40f9-bd71-ff2ddf53865c None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:48.150426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d4e50-d85b-4779-a410-2df8530db8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:48.151097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d4e50-d85b-4779-a410-2df8530db8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:48.151382 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d4e50-d85b-4779-a410-2df8530db8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:48.151654 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d4e50-d85b-4779-a410-2df8530db8f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:48.152468 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d4e50-d85b-4779-a410-2df8530db8f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:48.152771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d4e50-d85b-4779-a410-2df8530db8f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:48.154126 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d4e50-d85b-4779-a410-2df8530db8f3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:48.162822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:48.168920 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 507/1019] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:48.174743 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae96de-6fef-424b-89fd-7ee91d680049 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:48.285389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae96de-6fef-424b-89fd-7ee91d680049 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:48.309275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129db757-2867-49e0-a357-f058561847cc None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:48.309569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129db757-2867-49e0-a357-f058561847cc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:48.309842 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129db757-2867-49e0-a357-f058561847cc None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:48.311345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 508/1020] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:25:48.316796 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05941e-ca11-4532-9e2c-4a4b0e9d9d6a None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:48.317092 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05941e-ca11-4532-9e2c-4a4b0e9d9d6a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:48.317373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05941e-ca11-4532-9e2c-4a4b0e9d9d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:48.321098 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f85066-e18e-4c32-9ab9-461c3ed5369a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:48.321888 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f85066-e18e-4c32-9ab9-461c3ed5369a None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:48.322178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f85066-e18e-4c32-9ab9-461c3ed5369a None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:48.322453 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f85066-e18e-4c32-9ab9-461c3ed5369a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:48.331001 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f85066-e18e-4c32-9ab9-461c3ed5369a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:48.332472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f85066-e18e-4c32-9ab9-461c3ed5369a None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:48.334477 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-1e05941e-ca11-4532-9e2c-4a4b0e9d9d6a None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:48.335666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1e05941e-ca11-4532-9e2c-4a4b0e9d9d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:48.346806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-85f85066-e18e-4c32-9ab9-461c3ed5369a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 10:25:48.346806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:48.346806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:48.346806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:48.346806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:48.346806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:48.346806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:48.346806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:48.346806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:48.346806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:48.346806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:48.346806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:48.346806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:25:48.346806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:25:48.346806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:25:48.346806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:25:48.347994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:48.347994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:48.347994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:48.347994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:48.347994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:48.347994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:48.347994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:48.347994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:48.347994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:48.347994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:48.347994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:48.347994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:48.347994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:48.347994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:48.347994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:48.347994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:48.347994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:48.347994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:48.349162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:48.349162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:48.349162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:25:48.349162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:25:48.349162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:25:48.349162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:25:48.349162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:25:48.349162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:25:48.349162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 10:25:48.349162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:48.349162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 513/1021] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jun 6 10:25:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:48.353535 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f85066-e18e-4c32-9ab9-461c3ed5369a None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:48.357215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:48.419558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 509/1022] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:48 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 10:25:48.442791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05941e-ca11-4532-9e2c-4a4b0e9d9d6a None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:48.471367 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f85066-e18e-4c32-9ab9-461c3ed5369a None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:48.510722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf35cb5-00d9-470d-b77e-9e9ccdbbd0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:48.511389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf35cb5-00d9-470d-b77e-9e9ccdbbd0fd None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:48.511679 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf35cb5-00d9-470d-b77e-9e9ccdbbd0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:48.511988 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf35cb5-00d9-470d-b77e-9e9ccdbbd0fd None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:48.512990 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf35cb5-00d9-470d-b77e-9e9ccdbbd0fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:48.513299 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf35cb5-00d9-470d-b77e-9e9ccdbbd0fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:48.515486 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf35cb5-00d9-470d-b77e-9e9ccdbbd0fd None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:48.534847 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 514/1023] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jun 6 10:25:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:48.540144 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf35cb5-00d9-470d-b77e-9e9ccdbbd0fd None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:48.556804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05941e-ca11-4532-9e2c-4a4b0e9d9d6a None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:48.584106 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:48.592216 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12c95c-ac56-4118-a523-0e35e7dc1bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:48.592882 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12c95c-ac56-4118-a523-0e35e7dc1bc2 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:48.593167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12c95c-ac56-4118-a523-0e35e7dc1bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:48.593443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12c95c-ac56-4118-a523-0e35e7dc1bc2 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:48.594154 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12c95c-ac56-4118-a523-0e35e7dc1bc2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:48.596054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12c95c-ac56-4118-a523-0e35e7dc1bc2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:48.607867 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:48.611616 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 510/1024] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jun 6 10:25:48 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:25:48.654975 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf35cb5-00d9-470d-b77e-9e9ccdbbd0fd None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:48.702690 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67f173-2c47-4b9e-bf93-bcb39ef957ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:48.703342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67f173-2c47-4b9e-bf93-bcb39ef957ba None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:48.703620 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67f173-2c47-4b9e-bf93-bcb39ef957ba None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:48.704521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67f173-2c47-4b9e-bf93-bcb39ef957ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:48.782541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67f173-2c47-4b9e-bf93-bcb39ef957ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:48.784434 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67f173-2c47-4b9e-bf93-bcb39ef957ba None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:48.845963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:48.889527 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 515/1025] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:25:48 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 06 10:25:48.895921 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12c95c-ac56-4118-a523-0e35e7dc1bc2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:49.002321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12c95c-ac56-4118-a523-0e35e7dc1bc2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:49.037227 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c831f6-f148-4a3e-ae51-5f8dd6c45aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:49.038055 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c831f6-f148-4a3e-ae51-5f8dd6c45aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:49.038336 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c831f6-f148-4a3e-ae51-5f8dd6c45aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:49.038610 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c831f6-f148-4a3e-ae51-5f8dd6c45aa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:49.052179 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c831f6-f148-4a3e-ae51-5f8dd6c45aa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:49.053686 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c831f6-f148-4a3e-ae51-5f8dd6c45aa5 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:49.072743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-f1c831f6-f148-4a3e-ae51-5f8dd6c45aa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:49.072743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:49.072743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:49.072743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:49.072743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:49.072743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:49.072743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:49.072743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:49.072743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:49.072743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:49.072743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:49.072743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:49.072743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:49.072743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:49.072743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:49.073705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:49.073705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:49.073705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:49.073705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:49.073705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:49.073705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:49.073705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:49.073705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:49.073705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:49.073705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:49.073705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:49.073705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:49.073705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:49.073705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:49.073705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:49.073705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:49.073705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:49.074590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:49.074590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:49.074590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:49.074590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:49.076989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 511/1026] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:49.081416 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67f173-2c47-4b9e-bf93-bcb39ef957ba None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:49.188009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67f173-2c47-4b9e-bf93-bcb39ef957ba None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:49.222625 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97487ae0-f594-45a8-8c97-5337f90a2b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:49.223284 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97487ae0-f594-45a8-8c97-5337f90a2b20 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:49.223567 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97487ae0-f594-45a8-8c97-5337f90a2b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:49.223871 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97487ae0-f594-45a8-8c97-5337f90a2b20 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:49.224593 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97487ae0-f594-45a8-8c97-5337f90a2b20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:49.224901 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97487ae0-f594-45a8-8c97-5337f90a2b20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:49.226261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97487ae0-f594-45a8-8c97-5337f90a2b20 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:49.238230 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 516/1027] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:49.243563 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c831f6-f148-4a3e-ae51-5f8dd6c45aa5 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:49.289338 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:49.349486 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c831f6-f148-4a3e-ae51-5f8dd6c45aa5 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:49.373208 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bea85-f5fc-4884-87d3-c137edb81ac6 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:49.373667 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bea85-f5fc-4884-87d3-c137edb81ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:49.374018 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bea85-f5fc-4884-87d3-c137edb81ac6 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:49.375485 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 517/1028] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:49.380906 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861763c-1282-415e-9174-792e167ce9b8 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:49.381228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861763c-1282-415e-9174-792e167ce9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:49.381511 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861763c-1282-415e-9174-792e167ce9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:49.395637 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397811f2-2d2c-4ec4-afa8-891c27ddc6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:49.396545 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397811f2-2d2c-4ec4-afa8-891c27ddc6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:49.396828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397811f2-2d2c-4ec4-afa8-891c27ddc6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:49.397101 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397811f2-2d2c-4ec4-afa8-891c27ddc6e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:49.401805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-5861763c-1282-415e-9174-792e167ce9b8 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:49.403012 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-5861763c-1282-415e-9174-792e167ce9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:49.404470 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397811f2-2d2c-4ec4-afa8-891c27ddc6e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:49.405918 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397811f2-2d2c-4ec4-afa8-891c27ddc6e2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:49.417843 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:49.420867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-397811f2-2d2c-4ec4-afa8-891c27ddc6e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:49.420867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:49.420867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:49.420867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:49.420867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:49.420867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:49.420867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:49.420867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:49.420867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:49.420867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:49.420867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:49.420867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:49.420867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:49.420867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:49.420867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:49.421758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:49.421758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:49.421758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:49.421758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:49.421758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:49.421758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:49.421758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:49.421758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:49.421758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:49.421758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:49.421758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:49.421758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:49.421758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:49.421758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:49.421758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:49.421758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:49.421758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:49.423037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:49.423037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:49.423037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:49.423037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:49.423037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:49.423037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:49.423037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:49.423037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:49.423037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 512/1029] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:49.427705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397811f2-2d2c-4ec4-afa8-891c27ddc6e2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:49.482222 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 518/1030] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 10:25:49.505421 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861763c-1282-415e-9174-792e167ce9b8 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:49.531124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:49.534870 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397811f2-2d2c-4ec4-afa8-891c27ddc6e2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:49.570155 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a4f36-d6f1-43dd-86fc-0ba791b5335a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:49.570751 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a4f36-d6f1-43dd-86fc-0ba791b5335a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:49.571029 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a4f36-d6f1-43dd-86fc-0ba791b5335a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:49.571301 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a4f36-d6f1-43dd-86fc-0ba791b5335a None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:49.572196 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06a4f36-d6f1-43dd-86fc-0ba791b5335a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:49.572510 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06a4f36-d6f1-43dd-86fc-0ba791b5335a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:49.573938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06a4f36-d6f1-43dd-86fc-0ba791b5335a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:49.586857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 513/1031] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:25:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:49.592537 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a4f36-d6f1-43dd-86fc-0ba791b5335a None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:49.619593 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861763c-1282-415e-9174-792e167ce9b8 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:49.662075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae881f-0a06-4612-89d1-d95dbbe18169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:49.662927 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae881f-0a06-4612-89d1-d95dbbe18169 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:49.663207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae881f-0a06-4612-89d1-d95dbbe18169 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:49.663475 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae881f-0a06-4612-89d1-d95dbbe18169 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:49.664659 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae881f-0a06-4612-89d1-d95dbbe18169 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:49.666062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae881f-0a06-4612-89d1-d95dbbe18169 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:49.667661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 519/1032] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:49.673257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae881f-0a06-4612-89d1-d95dbbe18169 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:49.691016 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:49.699923 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a4f36-d6f1-43dd-86fc-0ba791b5335a None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:49.735290 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a745ad-ade3-4ece-a96a-c7943312eeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:49.736309 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a745ad-ade3-4ece-a96a-c7943312eeac None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:49.736903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a745ad-ade3-4ece-a96a-c7943312eeac None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:49.737352 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a745ad-ade3-4ece-a96a-c7943312eeac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:49.738381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a745ad-ade3-4ece-a96a-c7943312eeac None admin] RBAC: Adding query filter params (role.id=68d664d01d3340a58359d3fc9d738142, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=ed8a40e4f1034a8e99d0eb505e448db9) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:49.738929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a745ad-ade3-4ece-a96a-c7943312eeac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:49.740788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a745ad-ade3-4ece-a96a-c7943312eeac None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:49.756234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 514/1033] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:25:49 2022] GET /identity/v3/role_assignments?user.id=ed8a40e4f1034a8e99d0eb505e448db9&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=68d664d01d3340a58359d3fc9d738142 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:49.781197 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae881f-0a06-4612-89d1-d95dbbe18169 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:49.818468 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef8b5c-f510-4be5-b6ae-494fbf31ceb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:49.819152 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef8b5c-f510-4be5-b6ae-494fbf31ceb9 None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:49.819431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef8b5c-f510-4be5-b6ae-494fbf31ceb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:49.819703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef8b5c-f510-4be5-b6ae-494fbf31ceb9 None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:49.820528 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef8b5c-f510-4be5-b6ae-494fbf31ceb9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:49.821952 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef8b5c-f510-4be5-b6ae-494fbf31ceb9 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:49.854882 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 520/1034] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jun 6 10:25:49 2022] POST /identity/v3/projects => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:25:50.936375 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0456183-e955-4dda-81cf-24eec331191f None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:50.936890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0456183-e955-4dda-81cf-24eec331191f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:50.936968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0456183-e955-4dda-81cf-24eec331191f None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:50.938876 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 515/1035] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:50.944336 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64cbb08-04f1-427e-ab1a-f8dcc8688556 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:50.944756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64cbb08-04f1-427e-ab1a-f8dcc8688556 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:50.944889 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64cbb08-04f1-427e-ab1a-f8dcc8688556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:50.961661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-f64cbb08-04f1-427e-ab1a-f8dcc8688556 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:50.962833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-f64cbb08-04f1-427e-ab1a-f8dcc8688556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:51.028364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:51.035760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 521/1036] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:50 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:51.064902 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0456183-e955-4dda-81cf-24eec331191f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:51.091549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:51.172885 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0456183-e955-4dda-81cf-24eec331191f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:51.209756 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4a311-809d-4a67-ba59-a83837bf4253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:51.210471 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4a311-809d-4a67-ba59-a83837bf4253 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:51.210756 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4a311-809d-4a67-ba59-a83837bf4253 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:51.211034 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4a311-809d-4a67-ba59-a83837bf4253 None admin] PATH_INFO: `/v3/services/image` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:51.211681 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4a311-809d-4a67-ba59-a83837bf4253 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:51.213068 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4a311-809d-4a67-ba59-a83837bf4253 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:51.225086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-adb4a311-809d-4a67-ba59-a83837bf4253 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 10:25:51.225086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:51.225086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:51.225086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:51.225086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:51.225086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:51.225086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:51.225086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:51.225086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 10:25:51.225086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:51.225086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:51.225086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:51.225086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:51.225086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:51.225086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:51.225086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:51.225086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:51.226161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:51.226161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:51.226161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:51.226161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:51.226161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:51.226161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:51.226161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:51.226161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:51.226161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:51.226161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:51.226161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:51.226161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:51.226161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:51.226161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:51.226161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:51.226161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:51.227032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:51.227032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:51.227032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:51.227032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:51.227032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:51.227032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:51.227032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:51.227032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:51.227032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:51.227032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:51.227032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:51.227032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:51.227032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 10:25:51.227032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:51.227032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 516/1037] 10.222.0.42 () {56 vars in 1105 bytes} [Mon Jun 6 10:25:51 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:51.234031 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64cbb08-04f1-427e-ab1a-f8dcc8688556 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:51.337095 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64cbb08-04f1-427e-ab1a-f8dcc8688556 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:51.372063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd7d18-b10c-4247-90ee-81575f6211e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:51.372725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd7d18-b10c-4247-90ee-81575f6211e1 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:51.373007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd7d18-b10c-4247-90ee-81575f6211e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:51.373281 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd7d18-b10c-4247-90ee-81575f6211e1 None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:51.373981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd7d18-b10c-4247-90ee-81575f6211e1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:51.374284 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd7d18-b10c-4247-90ee-81575f6211e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:51.375731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd7d18-b10c-4247-90ee-81575f6211e1 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:51.387901 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 522/1038] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:51.390443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4a311-809d-4a67-ba59-a83837bf4253 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:51.480845 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:51.495759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4a311-809d-4a67-ba59-a83837bf4253 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:51.529099 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603da21-364e-42b5-94d3-3e50a91d1b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:51.529761 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603da21-364e-42b5-94d3-3e50a91d1b6d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:51.530041 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603da21-364e-42b5-94d3-3e50a91d1b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:51.530310 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603da21-364e-42b5-94d3-3e50a91d1b6d None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:51.531007 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603da21-364e-42b5-94d3-3e50a91d1b6d None admin] RBAC: Adding query filter params (type=image) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:51.531307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603da21-364e-42b5-94d3-3e50a91d1b6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:51.533380 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603da21-364e-42b5-94d3-3e50a91d1b6d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:51.538527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 517/1039] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:51 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:51.544123 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd7d18-b10c-4247-90ee-81575f6211e1 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:51.607992 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:51.649429 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd7d18-b10c-4247-90ee-81575f6211e1 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:51.686565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d367104-0f14-493e-99b8-c0cdd39c4d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:51.687236 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d367104-0f14-493e-99b8-c0cdd39c4d28 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:51.687516 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d367104-0f14-493e-99b8-c0cdd39c4d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:51.687864 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d367104-0f14-493e-99b8-c0cdd39c4d28 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:51.689192 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d367104-0f14-493e-99b8-c0cdd39c4d28 None admin] RBAC: Adding query filter params (interface=public, service_id=d38c6ec9625249e693e755ac56bbc851, region_id=RegionOne) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:51.689500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d367104-0f14-493e-99b8-c0cdd39c4d28 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:51.690902 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d367104-0f14-493e-99b8-c0cdd39c4d28 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:51.697130 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 523/1040] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jun 6 10:25:51 2022] GET /identity/v3/endpoints?service_id=d38c6ec9625249e693e755ac56bbc851&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:51.702495 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603da21-364e-42b5-94d3-3e50a91d1b6d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:51.803916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603da21-364e-42b5-94d3-3e50a91d1b6d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:51.838819 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f3885-3867-48cf-9264-f363b0a19bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:51.839511 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f3885-3867-48cf-9264-f363b0a19bec None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:51.839820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f3885-3867-48cf-9264-f363b0a19bec None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:51.840088 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f3885-3867-48cf-9264-f363b0a19bec None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:51.840779 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265f3885-3867-48cf-9264-f363b0a19bec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:51.842187 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265f3885-3867-48cf-9264-f363b0a19bec None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:51.849810 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 518/1041] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jun 6 10:25:51 2022] GET /identity/v3/services => generated 1957 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:25:52.010580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e947c4-09ad-4653-ab08-ee359d9d713f None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:52.011102 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e947c4-09ad-4653-ab08-ee359d9d713f None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:52.011102 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e947c4-09ad-4653-ab08-ee359d9d713f None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:52.012695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 524/1042] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:52.019181 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd405bc-1476-4fe3-8ecc-518cd60771c1 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:52.019697 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd405bc-1476-4fe3-8ecc-518cd60771c1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:52.019774 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd405bc-1476-4fe3-8ecc-518cd60771c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:52.037647 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-5bd405bc-1476-4fe3-8ecc-518cd60771c1 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:52.038834 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5bd405bc-1476-4fe3-8ecc-518cd60771c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:52.050953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:52.112351 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 519/1043] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:52.135356 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e947c4-09ad-4653-ab08-ee359d9d713f None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:52.180198 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b1891-ee8f-4d16-b10e-b3f074d07a9b None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:52.180521 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b1891-ee8f-4d16-b10e-b3f074d07a9b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:52.180752 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b1891-ee8f-4d16-b10e-b3f074d07a9b None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:52.182356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 520/1044] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:25:52.187307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9176d0b-02e7-4261-92ee-de09b8e11a9d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:52.187594 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9176d0b-02e7-4261-92ee-de09b8e11a9d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:52.188477 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9176d0b-02e7-4261-92ee-de09b8e11a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:52.206198 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-a9176d0b-02e7-4261-92ee-de09b8e11a9d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:52.207378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a9176d0b-02e7-4261-92ee-de09b8e11a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:52.245751 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e947c4-09ad-4653-ab08-ee359d9d713f None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:52.273695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:52.280817 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 521/1045] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 10:25:52.286256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56f8a0-0eb5-44bf-afcb-128d61fa77ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:52.287065 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56f8a0-0eb5-44bf-afcb-128d61fa77ea None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:52.287346 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56f8a0-0eb5-44bf-afcb-128d61fa77ea None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:52.287615 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56f8a0-0eb5-44bf-afcb-128d61fa77ea None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:52.298032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56f8a0-0eb5-44bf-afcb-128d61fa77ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:25:52.305356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9176d0b-02e7-4261-92ee-de09b8e11a9d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:52.306225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-dd56f8a0-0eb5-44bf-afcb-128d61fa77ea None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 10:25:52.306225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:52.306225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:52.306225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:52.306225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:52.306225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:52.306225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:52.306225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:52.306225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:52.306225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:52.306225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:52.306225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:52.306225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:25:52.306225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:25:52.306225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:25:52.306225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:25:52.307136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:25:52.307136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:25:52.307136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:25:52.307136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:25:52.307136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:52.307136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:52.307136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:52.307136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:52.307136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:52.307136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:52.307136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:52.307136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:52.307136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:52.307136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:52.307136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:52.307136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:52.308052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:52.308052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:52.308052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:52.308052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:52.308052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:25:52.308052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:25:52.308052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:25:52.308052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:25:52.308052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:25:52.308052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:25:52.308052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 10:25:52.308052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:52.308052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 525/1046] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Jun 6 10:25:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:52.314027 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56f8a0-0eb5-44bf-afcb-128d61fa77ea None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:52.390473 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:52.415642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9176d0b-02e7-4261-92ee-de09b8e11a9d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:52.424804 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56f8a0-0eb5-44bf-afcb-128d61fa77ea None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:52.453868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b00f4-0e04-4221-bcc1-2587a25f9677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:52.455090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b00f4-0e04-4221-bcc1-2587a25f9677 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:52.455377 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b00f4-0e04-4221-bcc1-2587a25f9677 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:52.455651 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b00f4-0e04-4221-bcc1-2587a25f9677 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:52.456869 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b00f4-0e04-4221-bcc1-2587a25f9677 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:52.458301 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b00f4-0e04-4221-bcc1-2587a25f9677 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:52.459939 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 522/1047] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:52.465498 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b00f4-0e04-4221-bcc1-2587a25f9677 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:52.468363 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbfce25-cc99-4e04-a7d2-e86f06943b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:52.469034 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfce25-cc99-4e04-a7d2-e86f06943b23 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:52.469322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfce25-cc99-4e04-a7d2-e86f06943b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:52.469636 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfce25-cc99-4e04-a7d2-e86f06943b23 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:52.470451 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbfce25-cc99-4e04-a7d2-e86f06943b23 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:52.470763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbfce25-cc99-4e04-a7d2-e86f06943b23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:52.472333 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbfce25-cc99-4e04-a7d2-e86f06943b23 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:52.481209 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 526/1048] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Jun 6 10:25:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:52.487643 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbfce25-cc99-4e04-a7d2-e86f06943b23 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:52.538502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:52.576120 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b00f4-0e04-4221-bcc1-2587a25f9677 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:52.595133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbfce25-cc99-4e04-a7d2-e86f06943b23 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:52.618705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67795d3-54a8-490f-9e98-1f4e8b709df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:52.619540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67795d3-54a8-490f-9e98-1f4e8b709df8 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:52.619851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67795d3-54a8-490f-9e98-1f4e8b709df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:52.620133 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67795d3-54a8-490f-9e98-1f4e8b709df8 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:52.620843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67795d3-54a8-490f-9e98-1f4e8b709df8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:52.622274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67795d3-54a8-490f-9e98-1f4e8b709df8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:52.634827 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df7fc0-1410-4b89-a8e7-bacb4bfca6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:52.635613 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df7fc0-1410-4b89-a8e7-bacb4bfca6da None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:52.635932 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df7fc0-1410-4b89-a8e7-bacb4bfca6da None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:52.636252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df7fc0-1410-4b89-a8e7-bacb4bfca6da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:52.640414 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:52.655822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 523/1049] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jun 6 10:25:52 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:25:52.656035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df7fc0-1410-4b89-a8e7-bacb4bfca6da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:52.656035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df7fc0-1410-4b89-a8e7-bacb4bfca6da None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:52.669925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-d4df7fc0-1410-4b89-a8e7-bacb4bfca6da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:52.669925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:52.669925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:52.669925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:52.669925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:52.669925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:52.669925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:52.669925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:52.669925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:52.669925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:52.669925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:52.669925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:52.669925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:52.669925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:52.669925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:52.671014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:52.671014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:52.671014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:52.671014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:52.671014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:52.671014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:52.671014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:52.671014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:52.671014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:52.671014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:52.671014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:52.671014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:52.671014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:52.671014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:52.671014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:52.671014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:52.671014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:52.673467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:52.673467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:52.673467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:52.673467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:52.673467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 527/1050] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:52.677920 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67795d3-54a8-490f-9e98-1f4e8b709df8 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:52.793139 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67795d3-54a8-490f-9e98-1f4e8b709df8 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:52.829501 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cbbcb-332f-4300-a0cd-78739df9e359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:52.830177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cbbcb-332f-4300-a0cd-78739df9e359 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:52.830461 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cbbcb-332f-4300-a0cd-78739df9e359 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:52.830736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cbbcb-332f-4300-a0cd-78739df9e359 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:52.831468 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cbbcb-332f-4300-a0cd-78739df9e359 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:52.831773 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cbbcb-332f-4300-a0cd-78739df9e359 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:52.833193 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cbbcb-332f-4300-a0cd-78739df9e359 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:52.851820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 524/1051] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:52.853174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df7fc0-1410-4b89-a8e7-bacb4bfca6da None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:52.957021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:52.960828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df7fc0-1410-4b89-a8e7-bacb4bfca6da None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:52.997481 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5aebcd-5932-4304-acbb-8428776927ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:52.998368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5aebcd-5932-4304-acbb-8428776927ae None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:52.998732 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5aebcd-5932-4304-acbb-8428776927ae None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:52.999084 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5aebcd-5932-4304-acbb-8428776927ae None admin] PATH_INFO: `/v3/users/nova` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:53.007411 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5aebcd-5932-4304-acbb-8428776927ae None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:53.008958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5aebcd-5932-4304-acbb-8428776927ae None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:53.023830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-cf5aebcd-5932-4304-acbb-8428776927ae None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 10:25:53.023830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:53.023830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:53.023830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:53.023830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:53.023830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:53.023830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:53.023830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:53.023830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:53.023830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:53.023830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:53.023830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:53.023830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:25:53.023830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:25:53.023830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:25:53.023830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:25:53.025227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:53.025227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:53.025227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:53.025227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:53.025227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:53.025227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:53.025227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:53.025227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:53.025227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:53.025227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:53.025227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:53.025227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:53.025227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:53.025227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:53.025227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:53.025227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:53.025227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:53.025227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:53.026167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:53.026167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:53.026167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:25:53.026167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:25:53.026167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:25:53.026167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:25:53.026167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:25:53.026167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:25:53.026167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 10:25:53.026167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:53.026167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 528/1052] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jun 6 10:25:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:53.030686 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cbbcb-332f-4300-a0cd-78739df9e359 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:53.091771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:53.134904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cbbcb-332f-4300-a0cd-78739df9e359 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:53.168773 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef41fe-31a5-4f2b-93c3-edfa053a92dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:53.169439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef41fe-31a5-4f2b-93c3-edfa053a92dd None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:53.169714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef41fe-31a5-4f2b-93c3-edfa053a92dd None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:53.169983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef41fe-31a5-4f2b-93c3-edfa053a92dd None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:53.170804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef41fe-31a5-4f2b-93c3-edfa053a92dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:53.171108 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef41fe-31a5-4f2b-93c3-edfa053a92dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:53.173227 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef41fe-31a5-4f2b-93c3-edfa053a92dd None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:53.195662 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 525/1053] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jun 6 10:25:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:53.197000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5aebcd-5932-4304-acbb-8428776927ae None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:53.302811 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5aebcd-5932-4304-acbb-8428776927ae None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:53.336424 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cc1be-da0f-4ef0-8eab-fa5c0e862fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:53.337207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cc1be-da0f-4ef0-8eab-fa5c0e862fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:53.337490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cc1be-da0f-4ef0-8eab-fa5c0e862fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:53.337761 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cc1be-da0f-4ef0-8eab-fa5c0e862fe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:53.348693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116cc1be-da0f-4ef0-8eab-fa5c0e862fe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:53.350187 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116cc1be-da0f-4ef0-8eab-fa5c0e862fe6 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:53.368674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-116cc1be-da0f-4ef0-8eab-fa5c0e862fe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:53.368674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:53.368674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:53.368674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:53.368674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:53.368674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:53.368674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:53.368674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:53.368674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:53.368674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:53.368674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:53.368674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:53.368674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:53.368674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:53.368674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:53.369571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:53.369571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:53.369571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:53.369571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:53.369571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:53.369571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:53.369571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:53.369571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:53.369571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:53.369571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:53.369571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:53.369571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:53.369571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:53.369571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:53.369571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:53.369571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:53.369571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:53.370468 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:53.370468 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:53.370468 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:53.370468 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:53.370468 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 529/1054] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:53.375590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef41fe-31a5-4f2b-93c3-edfa053a92dd None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:53.484139 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef41fe-31a5-4f2b-93c3-edfa053a92dd None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:53.502300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:53.520791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe89ac-1cb8-4df1-a316-723885578a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:53.521459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe89ac-1cb8-4df1-a316-723885578a14 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:53.521745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe89ac-1cb8-4df1-a316-723885578a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:53.522023 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe89ac-1cb8-4df1-a316-723885578a14 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:53.523000 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe89ac-1cb8-4df1-a316-723885578a14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:53.523309 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe89ac-1cb8-4df1-a316-723885578a14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:53.524698 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe89ac-1cb8-4df1-a316-723885578a14 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:53.540796 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 526/1055] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:53.542335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cc1be-da0f-4ef0-8eab-fa5c0e862fe6 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:53.558310 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:53.648261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cc1be-da0f-4ef0-8eab-fa5c0e862fe6 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:53.682146 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e7fee9-6dd5-4900-bc40-cd1a832f986f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:53.683002 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7fee9-6dd5-4900-bc40-cd1a832f986f None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:53.683290 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7fee9-6dd5-4900-bc40-cd1a832f986f None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:53.683566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7fee9-6dd5-4900-bc40-cd1a832f986f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:53.691218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e7fee9-6dd5-4900-bc40-cd1a832f986f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:53.692760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e7fee9-6dd5-4900-bc40-cd1a832f986f None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:53.708265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-b6e7fee9-6dd5-4900-bc40-cd1a832f986f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:53.708265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:53.708265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:53.708265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:53.708265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:53.708265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:53.708265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:53.708265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:53.708265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:53.708265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:53.708265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:53.708265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:53.708265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:53.708265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:53.708265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:53.709193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:53.709193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:53.709193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:53.709193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:53.709193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:53.709193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:53.709193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:53.709193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:53.709193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:53.709193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:53.709193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:53.709193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:53.709193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:53.709193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:53.709193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:53.709193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:53.709193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:53.710113 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:53.710113 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:53.710113 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:53.710113 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:53.710113 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:53.710113 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:53.710113 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:53.710113 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:53.711284 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 530/1056] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:53.715596 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe89ac-1cb8-4df1-a316-723885578a14 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:53.831523 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe89ac-1cb8-4df1-a316-723885578a14 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:53.872280 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c011d6a-3ec1-41c2-a7d0-548189752a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:53.872280 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c011d6a-3ec1-41c2-a7d0-548189752a97 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:53.872280 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c011d6a-3ec1-41c2-a7d0-548189752a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:53.872280 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c011d6a-3ec1-41c2-a7d0-548189752a97 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:53.872718 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c011d6a-3ec1-41c2-a7d0-548189752a97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:53.872718 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c011d6a-3ec1-41c2-a7d0-548189752a97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:53.874059 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c011d6a-3ec1-41c2-a7d0-548189752a97 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:53.895828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 527/1057] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:25:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:53.898036 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e7fee9-6dd5-4900-bc40-cd1a832f986f None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:53.973672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:54.007223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e7fee9-6dd5-4900-bc40-cd1a832f986f None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:54.048650 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e855e0-81b7-4dc6-a0bf-9ed7523a72f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:54.050133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e855e0-81b7-4dc6-a0bf-9ed7523a72f8 None admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:54.050433 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e855e0-81b7-4dc6-a0bf-9ed7523a72f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:54.050703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e855e0-81b7-4dc6-a0bf-9ed7523a72f8 None admin] PATH_INFO: `/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/ed8a40e4f1034a8e99d0eb505e448db9/roles/68d664d01d3340a58359d3fc9d738142` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:54.053734 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e855e0-81b7-4dc6-a0bf-9ed7523a72f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f082d7ed3694fd9998623f3d4a5fea9, user_id=ed8a40e4f1034a8e99d0eb505e448db9, role_id=68d664d01d3340a58359d3fc9d738142)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:54.055910 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e855e0-81b7-4dc6-a0bf-9ed7523a72f8 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:54.075826 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 531/1058] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:25:53 2022] PUT /identity/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/ed8a40e4f1034a8e99d0eb505e448db9/roles/68d664d01d3340a58359d3fc9d738142 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:25:54.186727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc17f0-9945-41c4-b947-cc239eccfd21 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:54.187255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc17f0-9945-41c4-b947-cc239eccfd21 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:54.187493 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc17f0-9945-41c4-b947-cc239eccfd21 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:54.190481 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 528/1059] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:54.196550 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a11bc-4459-4759-a70a-4ae13d9bc769 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:54.196891 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a11bc-4459-4759-a70a-4ae13d9bc769 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:54.197176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a11bc-4459-4759-a70a-4ae13d9bc769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:54.217660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-477a11bc-4459-4759-a70a-4ae13d9bc769 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:54.218872 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-477a11bc-4459-4759-a70a-4ae13d9bc769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:54.301571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 532/1060] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:54.328212 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc17f0-9945-41c4-b947-cc239eccfd21 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:54.391384 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:54.466254 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc17f0-9945-41c4-b947-cc239eccfd21 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:54.505218 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11aa71-3dfb-435d-9811-c3c7c2cebc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:54.506012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11aa71-3dfb-435d-9811-c3c7c2cebc76 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:54.506301 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11aa71-3dfb-435d-9811-c3c7c2cebc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:54.506566 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11aa71-3dfb-435d-9811-c3c7c2cebc76 None admin] PATH_INFO: `/v3/services/image` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:54.507223 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11aa71-3dfb-435d-9811-c3c7c2cebc76 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:54.508632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11aa71-3dfb-435d-9811-c3c7c2cebc76 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:54.523712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-ad11aa71-3dfb-435d-9811-c3c7c2cebc76 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 10:25:54.523712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:54.523712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:25:54.523712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:25:54.523712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:25:54.523712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:25:54.523712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:25:54.523712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:54.523712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 10:25:54.523712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:54.523712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:25:54.523712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:54.523712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:54.523712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:54.523712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:54.523712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:54.523712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:54.524853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:54.524853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:54.524853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:54.524853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:54.524853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:54.524853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:54.524853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:25:54.524853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:25:54.524853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:25:54.524853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:25:54.524853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:54.524853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:54.524853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:54.524853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:54.524853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:54.524853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:54.525735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:54.525735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:54.525735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:54.525735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:54.525735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:54.525735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:54.525735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:54.525735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:54.525735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:54.525735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:54.525735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:25:54.525735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:25:54.525735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 10:25:54.525735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:54.527882 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 529/1061] 10.222.0.42 () {56 vars in 1105 bytes} [Mon Jun 6 10:25:54 2022] GET /identity/v3/services/image => generated 86 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:54.531413 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a11bc-4459-4759-a70a-4ae13d9bc769 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:54.603329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:54.652374 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a11bc-4459-4759-a70a-4ae13d9bc769 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:54.689442 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e137d1d-d028-49cb-aff4-2a4ff0122f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:54.690124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e137d1d-d028-49cb-aff4-2a4ff0122f31 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:54.690405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e137d1d-d028-49cb-aff4-2a4ff0122f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:54.690679 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e137d1d-d028-49cb-aff4-2a4ff0122f31 None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:54.691388 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e137d1d-d028-49cb-aff4-2a4ff0122f31 None admin] RBAC: Adding query filter params (name=image) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:54.691693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e137d1d-d028-49cb-aff4-2a4ff0122f31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:54.693094 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e137d1d-d028-49cb-aff4-2a4ff0122f31 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:54.704424 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 533/1062] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:54.706080 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11aa71-3dfb-435d-9811-c3c7c2cebc76 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:54.821603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11aa71-3dfb-435d-9811-c3c7c2cebc76 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:54.823267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:54.858813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131fd80-9abe-4f36-8913-3de842156356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:54.859481 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131fd80-9abe-4f36-8913-3de842156356 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:54.859956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131fd80-9abe-4f36-8913-3de842156356 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:54.860241 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131fd80-9abe-4f36-8913-3de842156356 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:54.861057 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131fd80-9abe-4f36-8913-3de842156356 None admin] RBAC: Adding query filter params (type=image) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:54.861367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131fd80-9abe-4f36-8913-3de842156356 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:54.862752 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131fd80-9abe-4f36-8913-3de842156356 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:54.872536 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 530/1063] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:54.873561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e137d1d-d028-49cb-aff4-2a4ff0122f31 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:54.985522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e137d1d-d028-49cb-aff4-2a4ff0122f31 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:55.011700 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:55.022023 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b086f-98de-4891-b9a0-a9b78e75775b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:55.022938 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b086f-98de-4891-b9a0-a9b78e75775b None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:55.023223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b086f-98de-4891-b9a0-a9b78e75775b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:55.023501 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b086f-98de-4891-b9a0-a9b78e75775b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:55.024272 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b086f-98de-4891-b9a0-a9b78e75775b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:55.025634 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b086f-98de-4891-b9a0-a9b78e75775b None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:55.044131 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 534/1064] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jun 6 10:25:54 2022] POST /identity/v3/endpoints => generated 327 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:25:55.142523 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806e39e-cf64-4e25-9946-9f216b95dc38 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:55.142723 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806e39e-cf64-4e25-9946-9f216b95dc38 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:55.142990 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806e39e-cf64-4e25-9946-9f216b95dc38 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:55.144586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 531/1065] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:55.150502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140367c2-fe2f-44e3-8ab6-552580ff742e None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:55.151038 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140367c2-fe2f-44e3-8ab6-552580ff742e None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:55.151130 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140367c2-fe2f-44e3-8ab6-552580ff742e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:55.169198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-140367c2-fe2f-44e3-8ab6-552580ff742e None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:55.170405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-140367c2-fe2f-44e3-8ab6-552580ff742e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:55.256028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 535/1066] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:25:55.280342 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806e39e-cf64-4e25-9946-9f216b95dc38 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:55.399130 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806e39e-cf64-4e25-9946-9f216b95dc38 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:55.433995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005cb01-a5fb-4398-a402-4f6fec876db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:55.434798 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005cb01-a5fb-4398-a402-4f6fec876db4 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:55.435083 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005cb01-a5fb-4398-a402-4f6fec876db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:55.435360 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005cb01-a5fb-4398-a402-4f6fec876db4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:55.446238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005cb01-a5fb-4398-a402-4f6fec876db4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:25:55.454887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-c005cb01-a5fb-4398-a402-4f6fec876db4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:25:55.454887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:55.454887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:55.454887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:55.454887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:55.454887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:55.454887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:55.454887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:55.454887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:55.454887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:55.454887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:55.454887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:55.454887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:25:55.454887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:25:55.454887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:25:55.454887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:25:55.456097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:25:55.456097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:25:55.456097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:25:55.456097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:25:55.456097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:55.456097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:55.456097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:55.456097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:55.456097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:55.456097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:55.456097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:55.456097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:55.456097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:55.456097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:55.456097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:55.456097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:55.456973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:55.456973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:55.456973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:55.456973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:55.456973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:25:55.456973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:25:55.456973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:25:55.456973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:25:55.456973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:25:55.456973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:25:55.456973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:25:55.456973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:55.456973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 532/1067] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:25:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:55.462069 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140367c2-fe2f-44e3-8ab6-552580ff742e None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:55.559223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140367c2-fe2f-44e3-8ab6-552580ff742e None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:55.568464 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:55.595647 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230ebf9-d821-4928-8fe1-d5990165f2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:55.596664 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230ebf9-d821-4928-8fe1-d5990165f2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:55.597154 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230ebf9-d821-4928-8fe1-d5990165f2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:55.597661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230ebf9-d821-4928-8fe1-d5990165f2c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:55.598603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230ebf9-d821-4928-8fe1-d5990165f2c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:55.599113 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230ebf9-d821-4928-8fe1-d5990165f2c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:55.600697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230ebf9-d821-4928-8fe1-d5990165f2c7 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:55.610700 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 536/1068] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:25:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:55.615992 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005cb01-a5fb-4398-a402-4f6fec876db4 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:55.650708 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:55.734129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005cb01-a5fb-4398-a402-4f6fec876db4 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:55.769321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864548b-93fb-40ca-aabc-3499b669682f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:55.770197 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864548b-93fb-40ca-aabc-3499b669682f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:55.770473 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864548b-93fb-40ca-aabc-3499b669682f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:55.770739 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864548b-93fb-40ca-aabc-3499b669682f None admin] PATH_INFO: `/v3/users/bde997d1a2004a99890991705714a627` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:55.784471 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864548b-93fb-40ca-aabc-3499b669682f None admin] RBAC: Authorizing `identity:get_user(user_id=bde997d1a2004a99890991705714a627)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:55.785909 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864548b-93fb-40ca-aabc-3499b669682f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:55.787811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 533/1069] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:25:55 2022] GET /identity/v3/users/bde997d1a2004a99890991705714a627 => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:55.793231 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230ebf9-d821-4928-8fe1-d5990165f2c7 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:55.894509 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230ebf9-d821-4928-8fe1-d5990165f2c7 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:55.928724 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06091e73-2893-406f-9636-4c6784fe0b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:55.929286 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06091e73-2893-406f-9636-4c6784fe0b6f None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:55.929565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06091e73-2893-406f-9636-4c6784fe0b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:55.929836 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06091e73-2893-406f-9636-4c6784fe0b6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:55.968331 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:55.998682 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06091e73-2893-406f-9636-4c6784fe0b6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:56.000149 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06091e73-2893-406f-9636-4c6784fe0b6f None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:56.118141 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 537/1070] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:25:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 10:25:56.119916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864548b-93fb-40ca-aabc-3499b669682f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:56.203193 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:56.221158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864548b-93fb-40ca-aabc-3499b669682f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:56.256733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efb425-4be9-4be3-a4db-b09b6aa5544d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:56.257598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efb425-4be9-4be3-a4db-b09b6aa5544d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:56.257876 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efb425-4be9-4be3-a4db-b09b6aa5544d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:56.258146 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efb425-4be9-4be3-a4db-b09b6aa5544d None admin] PATH_INFO: `/v3/projects/a67b912ec01746b9ba9657ba9afd5901` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:56.259313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efb425-4be9-4be3-a4db-b09b6aa5544d None admin] RBAC: Authorizing `identity:get_project(project_id=a67b912ec01746b9ba9657ba9afd5901)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:56.260756 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efb425-4be9-4be3-a4db-b09b6aa5544d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:56.267842 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 534/1071] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:25:56 2022] GET /identity/v3/projects/a67b912ec01746b9ba9657ba9afd5901 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:56.269658 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06091e73-2893-406f-9636-4c6784fe0b6f None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:56.346257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:56.386545 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06091e73-2893-406f-9636-4c6784fe0b6f None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:56.427068 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcc10e-519c-4309-8cfc-97c57ca7afb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:56.427725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcc10e-519c-4309-8cfc-97c57ca7afb0 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:56.428037 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcc10e-519c-4309-8cfc-97c57ca7afb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:56.428313 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcc10e-519c-4309-8cfc-97c57ca7afb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:56.429143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcc10e-519c-4309-8cfc-97c57ca7afb0 None admin] RBAC: Adding query filter params (role.id=0f929204eb7b4ea8859338ed2d3ff75b, scope.project.id=a67b912ec01746b9ba9657ba9afd5901, user.id=bde997d1a2004a99890991705714a627) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:56.429455 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcc10e-519c-4309-8cfc-97c57ca7afb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:56.430892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcc10e-519c-4309-8cfc-97c57ca7afb0 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:56.434208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3dfd57-af8c-411c-a106-7478063d60ca None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:56.434652 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3dfd57-af8c-411c-a106-7478063d60ca None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:56.434744 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3dfd57-af8c-411c-a106-7478063d60ca None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:56.436288 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 535/1072] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:56.443869 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 538/1073] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:25:56 2022] GET /identity/v3/role_assignments?user.id=bde997d1a2004a99890991705714a627&scope.project.id=a67b912ec01746b9ba9657ba9afd5901&role.id=0f929204eb7b4ea8859338ed2d3ff75b => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:56.444099 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fd273-adfa-4207-99b6-fad4772d4de1 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:56.444099 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fd273-adfa-4207-99b6-fad4772d4de1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:56.444099 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fd273-adfa-4207-99b6-fad4772d4de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:56.460009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-7f6fd273-adfa-4207-99b6-fad4772d4de1 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:56.461137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-7f6fd273-adfa-4207-99b6-fad4772d4de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:56.537515 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 536/1074] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 10:25:56.561147 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcc10e-519c-4309-8cfc-97c57ca7afb0 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:56.671066 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcc10e-519c-4309-8cfc-97c57ca7afb0 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:56.709666 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f8f0c-1c1b-4992-8475-476f2c3a410c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:56.710534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f8f0c-1c1b-4992-8475-476f2c3a410c None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:56.710757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f8f0c-1c1b-4992-8475-476f2c3a410c None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:56.711027 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f8f0c-1c1b-4992-8475-476f2c3a410c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:56.712885 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:56.722811 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f8f0c-1c1b-4992-8475-476f2c3a410c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:25:56.731015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-c22f8f0c-1c1b-4992-8475-476f2c3a410c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 10:25:56.731015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:56.731015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:56.731015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:56.731015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:56.731015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:56.731015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:56.731015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:56.731015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:56.731015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:56.731015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:56.731015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:56.731015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:25:56.731015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:25:56.731015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:25:56.731015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:25:56.731997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:25:56.731997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:25:56.731997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:25:56.731997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:25:56.731997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:56.731997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:56.731997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:56.731997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:56.731997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:56.731997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:56.731997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:56.731997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:56.731997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:56.731997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:56.731997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:56.731997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:56.732851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:56.732851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:56.732851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:56.732851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:56.732851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:25:56.732851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:25:56.732851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:25:56.732851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:25:56.732851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:25:56.732851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:25:56.732851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 10:25:56.732851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:56.732851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 539/1075] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Jun 6 10:25:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:56.738152 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fd273-adfa-4207-99b6-fad4772d4de1 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:56.821377 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:56.841076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fd273-adfa-4207-99b6-fad4772d4de1 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:56.876876 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d225fb4-9b85-430e-af5d-fb59e933ca65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:56.877546 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d225fb4-9b85-430e-af5d-fb59e933ca65 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:56.877828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d225fb4-9b85-430e-af5d-fb59e933ca65 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:56.878098 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d225fb4-9b85-430e-af5d-fb59e933ca65 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:56.878845 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d225fb4-9b85-430e-af5d-fb59e933ca65 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:56.879146 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d225fb4-9b85-430e-af5d-fb59e933ca65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:56.880534 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d225fb4-9b85-430e-af5d-fb59e933ca65 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:56.893521 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 537/1076] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Jun 6 10:25:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:56.894916 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f8f0c-1c1b-4992-8475-476f2c3a410c None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:56.997648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f8f0c-1c1b-4992-8475-476f2c3a410c None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:57.033164 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac3d6d-2dee-41cd-add8-493ef458b981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:57.033953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac3d6d-2dee-41cd-add8-493ef458b981 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:57.034236 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac3d6d-2dee-41cd-add8-493ef458b981 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:57.034511 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac3d6d-2dee-41cd-add8-493ef458b981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:57.045175 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac3d6d-2dee-41cd-add8-493ef458b981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:57.046660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac3d6d-2dee-41cd-add8-493ef458b981 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:57.063656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-eeac3d6d-2dee-41cd-add8-493ef458b981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:57.063656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:57.063656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:57.063656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:57.063656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:57.063656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:57.063656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:57.063656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:57.063656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:57.063656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:57.063656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:57.063656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:57.063656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:57.063656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:57.063656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:57.064604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:57.064604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:57.064604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:57.064604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:57.064604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:57.064604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:57.064604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:57.064604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:57.064604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:57.064604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:57.064604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:57.064604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:57.064604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:57.064604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:57.064604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:57.064604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:57.064604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:57.065922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:57.065922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:57.065922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:57.065922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:57.065922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 540/1077] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:57.070531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d225fb4-9b85-430e-af5d-fb59e933ca65 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:57.182457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d225fb4-9b85-430e-af5d-fb59e933ca65 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:57.217000 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411944c5-63f5-4de1-8e14-4dc125fb3ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:57.217662 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411944c5-63f5-4de1-8e14-4dc125fb3ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:57.217945 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411944c5-63f5-4de1-8e14-4dc125fb3ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:57.218221 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411944c5-63f5-4de1-8e14-4dc125fb3ac1 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:57.219547 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411944c5-63f5-4de1-8e14-4dc125fb3ac1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:57.219894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411944c5-63f5-4de1-8e14-4dc125fb3ac1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:57.221260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411944c5-63f5-4de1-8e14-4dc125fb3ac1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:57.237750 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 538/1078] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:57.239021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac3d6d-2dee-41cd-add8-493ef458b981 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:57.290288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:57.347135 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac3d6d-2dee-41cd-add8-493ef458b981 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:57.388499 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3338d6-2a26-4fc4-81f4-36ee39a06d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:57.389473 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3338d6-2a26-4fc4-81f4-36ee39a06d04 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:57.389929 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3338d6-2a26-4fc4-81f4-36ee39a06d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:57.390349 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3338d6-2a26-4fc4-81f4-36ee39a06d04 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:57.398798 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3338d6-2a26-4fc4-81f4-36ee39a06d04 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:57.400432 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3338d6-2a26-4fc4-81f4-36ee39a06d04 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:57.414810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-ac3338d6-2a26-4fc4-81f4-36ee39a06d04 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 10:25:57.414810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:57.414810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:57.414810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:57.414810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:57.414810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:57.414810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:57.414810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:57.414810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:57.414810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:57.414810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:57.414810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:57.414810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:25:57.414810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:25:57.414810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:25:57.414810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:25:57.415858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:57.415858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:57.415858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:25:57.415858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:57.415858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:25:57.415858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:25:57.415858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:57.415858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:57.415858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:57.415858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:57.415858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:57.415858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:57.415858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:57.415858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:57.415858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:57.415858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:57.415858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:57.415858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:57.416809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:57.416809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:57.416809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:25:57.416809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:25:57.416809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:25:57.416809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:25:57.416809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:25:57.416809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:25:57.416809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 10:25:57.416809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:57.417350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 541/1079] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jun 6 10:25:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:57.423637 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411944c5-63f5-4de1-8e14-4dc125fb3ac1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:57.428452 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:57.434543 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5944fc-485f-4470-aae3-8c84decbc27b None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:57.434831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5944fc-485f-4470-aae3-8c84decbc27b None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:57.435108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5944fc-485f-4470-aae3-8c84decbc27b None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:57.436613 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 542/1080] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:25:57.441956 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629eaca4-510e-4d93-9c39-341a4723d504 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:57.442542 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629eaca4-510e-4d93-9c39-341a4723d504 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:57.442542 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629eaca4-510e-4d93-9c39-341a4723d504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:57.460801 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-629eaca4-510e-4d93-9c39-341a4723d504 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:57.461974 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-629eaca4-510e-4d93-9c39-341a4723d504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:57.536703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411944c5-63f5-4de1-8e14-4dc125fb3ac1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:57.539676 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 543/1081] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 10:25:57.562256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629eaca4-510e-4d93-9c39-341a4723d504 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:57.578078 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592221e-3cc4-47f0-9264-bdd2ff2b719f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:57.578997 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592221e-3cc4-47f0-9264-bdd2ff2b719f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:57.579291 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592221e-3cc4-47f0-9264-bdd2ff2b719f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:57.579571 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592221e-3cc4-47f0-9264-bdd2ff2b719f None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:57.580411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592221e-3cc4-47f0-9264-bdd2ff2b719f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:57.580780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592221e-3cc4-47f0-9264-bdd2ff2b719f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:57.582138 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592221e-3cc4-47f0-9264-bdd2ff2b719f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:57.600012 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:57.605262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 539/1082] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jun 6 10:25:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:57.606377 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592221e-3cc4-47f0-9264-bdd2ff2b719f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:57.670953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629eaca4-510e-4d93-9c39-341a4723d504 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:57.674385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:57.706280 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932f527-d215-4da2-9493-010833cfc59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:57.706936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932f527-d215-4da2-9493-010833cfc59e None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:57.707222 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932f527-d215-4da2-9493-010833cfc59e None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:57.707503 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932f527-d215-4da2-9493-010833cfc59e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:57.718592 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592221e-3cc4-47f0-9264-bdd2ff2b719f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:57.755553 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307fd8e-532c-44ca-9da9-244cea46ac2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:57.756349 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307fd8e-532c-44ca-9da9-244cea46ac2d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:57.756654 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307fd8e-532c-44ca-9da9-244cea46ac2d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:57.757025 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307fd8e-532c-44ca-9da9-244cea46ac2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:57.775921 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932f527-d215-4da2-9493-010833cfc59e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:57.777365 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932f527-d215-4da2-9493-010833cfc59e None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:57.813170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:57.831656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307fd8e-532c-44ca-9da9-244cea46ac2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:57.833146 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307fd8e-532c-44ca-9da9-244cea46ac2d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:57.886922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 544/1083] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:25:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 06 10:25:57.887990 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932f527-d215-4da2-9493-010833cfc59e None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:57.896949 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:57.940911 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 540/1084] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:25:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 10:25:57.946372 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307fd8e-532c-44ca-9da9-244cea46ac2d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:57.992022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932f527-d215-4da2-9493-010833cfc59e None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:58.027264 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc73579-145c-4ff0-8c1e-30984b9e1268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:58.028081 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc73579-145c-4ff0-8c1e-30984b9e1268 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:58.028363 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc73579-145c-4ff0-8c1e-30984b9e1268 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:58.028638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc73579-145c-4ff0-8c1e-30984b9e1268 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:58.032307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:58.043456 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc73579-145c-4ff0-8c1e-30984b9e1268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:58.044932 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc73579-145c-4ff0-8c1e-30984b9e1268 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:58.053849 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307fd8e-532c-44ca-9da9-244cea46ac2d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:58.063080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-edc73579-145c-4ff0-8c1e-30984b9e1268 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:58.063080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:58.063080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:58.063080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:58.063080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:58.063080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:58.063080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:58.063080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:58.063080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:58.063080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:58.063080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:58.063080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:58.063080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:58.063080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:58.063080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:58.064186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:58.064186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:58.064186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:58.064186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:58.064186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:58.064186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:58.064186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:58.064186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:58.064186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:58.064186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:58.064186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:58.064186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:58.064186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:58.064186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:58.064186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:58.064186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:58.064186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:58.065186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:58.065186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:58.065186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:58.065186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:58.065186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 545/1085] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:58.070260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc73579-145c-4ff0-8c1e-30984b9e1268 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:58.093126 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deda133-2c8e-4e99-b569-e720b18e9651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:58.093960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deda133-2c8e-4e99-b569-e720b18e9651 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:58.094245 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deda133-2c8e-4e99-b569-e720b18e9651 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:58.094818 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deda133-2c8e-4e99-b569-e720b18e9651 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:58.104945 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deda133-2c8e-4e99-b569-e720b18e9651 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:58.106399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deda133-2c8e-4e99-b569-e720b18e9651 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:58.124968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-7deda133-2c8e-4e99-b569-e720b18e9651 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:58.124968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:58.124968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:58.124968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:58.124968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:58.124968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:58.124968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:58.124968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:58.124968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:58.124968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:58.124968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:58.124968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:58.124968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:25:58.124968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:25:58.124968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:25:58.126611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:25:58.126611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:58.126611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:58.126611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:58.126611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:58.126611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:58.126611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:58.126611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:58.126611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:58.126611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:58.126611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:58.126611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:58.126611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:58.126611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:58.126611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:58.126611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:58.126611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:58.132419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:25:58.132419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:25:58.132419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:25:58.132419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:25:58.132419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 541/1086] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:25:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:25:58.132419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deda133-2c8e-4e99-b569-e720b18e9651 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:58.149175 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:58.183806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc73579-145c-4ff0-8c1e-30984b9e1268 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:58.220602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c62b15-fb31-4f5a-bf8f-c11a878e9cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:58.221288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c62b15-fb31-4f5a-bf8f-c11a878e9cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:58.221573 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c62b15-fb31-4f5a-bf8f-c11a878e9cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:58.221846 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c62b15-fb31-4f5a-bf8f-c11a878e9cd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:58.222567 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c62b15-fb31-4f5a-bf8f-c11a878e9cd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:58.222861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c62b15-fb31-4f5a-bf8f-c11a878e9cd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:58.224277 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c62b15-fb31-4f5a-bf8f-c11a878e9cd3 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:58.247813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 546/1087] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:58.248029 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deda133-2c8e-4e99-b569-e720b18e9651 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:58.277160 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69aacc-a745-4cbd-8990-fd89ec7cf5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:58.277837 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69aacc-a745-4cbd-8990-fd89ec7cf5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:58.278114 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69aacc-a745-4cbd-8990-fd89ec7cf5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:58.278384 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69aacc-a745-4cbd-8990-fd89ec7cf5c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:58.279147 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69aacc-a745-4cbd-8990-fd89ec7cf5c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:58.279451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69aacc-a745-4cbd-8990-fd89ec7cf5c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:58.280880 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69aacc-a745-4cbd-8990-fd89ec7cf5c3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:58.293657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 542/1088] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:25:58.299424 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c62b15-fb31-4f5a-bf8f-c11a878e9cd3 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:58.354900 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:58.418863 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c62b15-fb31-4f5a-bf8f-c11a878e9cd3 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:58.459523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936af62-b507-4601-803a-47e483e7b08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:58.460373 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936af62-b507-4601-803a-47e483e7b08f None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:58.460870 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936af62-b507-4601-803a-47e483e7b08f None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:58.460870 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936af62-b507-4601-803a-47e483e7b08f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:58.468782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936af62-b507-4601-803a-47e483e7b08f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:58.470463 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936af62-b507-4601-803a-47e483e7b08f None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:58.486782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-3936af62-b507-4601-803a-47e483e7b08f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:58.486782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:58.486782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:58.486782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:58.486782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:58.486782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:58.486782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:58.486782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:58.486782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:58.486782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:58.486782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:58.486782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:58.486782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:25:58.486782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:25:58.486782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:25:58.487729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:25:58.487729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:58.487729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:58.487729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:58.487729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:58.487729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:58.487729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:58.487729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:58.487729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:58.487729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:58.487729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:58.487729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:58.487729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:58.487729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:58.487729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:58.487729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:58.487729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:58.488731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:25:58.488731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:25:58.488731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:25:58.488731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:25:58.488731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:25:58.488731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:25:58.488731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:25:58.488731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:58.488731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 547/1089] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:25:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:58.495918 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69aacc-a745-4cbd-8990-fd89ec7cf5c3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:58.576987 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:58.603537 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69aacc-a745-4cbd-8990-fd89ec7cf5c3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:58.636375 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c56066-ae5e-483b-81b7-260bd9f30a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:58.636956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c56066-ae5e-483b-81b7-260bd9f30a7e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:58.637234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c56066-ae5e-483b-81b7-260bd9f30a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:58.637505 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c56066-ae5e-483b-81b7-260bd9f30a7e None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:58.638280 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c56066-ae5e-483b-81b7-260bd9f30a7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:58.638584 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c56066-ae5e-483b-81b7-260bd9f30a7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:58.640012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c56066-ae5e-483b-81b7-260bd9f30a7e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:58.652004 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 543/1090] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:25:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:58.658193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936af62-b507-4601-803a-47e483e7b08f None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:58.721288 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269f559-6601-4577-987d-0dc6107f9c04 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:58.721572 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269f559-6601-4577-987d-0dc6107f9c04 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:58.721844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269f559-6601-4577-987d-0dc6107f9c04 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:58.723204 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 544/1091] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:25:58.728052 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b94794-3a29-450e-a62f-412a3a2dbbc7 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:58.728333 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b94794-3a29-450e-a62f-412a3a2dbbc7 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:58.728606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b94794-3a29-450e-a62f-412a3a2dbbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:58.745036 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b0b94794-3a29-450e-a62f-412a3a2dbbc7 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:25:58.746224 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b0b94794-3a29-450e-a62f-412a3a2dbbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:25:58.757606 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936af62-b507-4601-803a-47e483e7b08f None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:58.766693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:58.792664 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abd0e3-1928-48ec-9590-79c4015d5cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:58.793300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abd0e3-1928-48ec-9590-79c4015d5cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:58.793577 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abd0e3-1928-48ec-9590-79c4015d5cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:58.793845 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abd0e3-1928-48ec-9590-79c4015d5cb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:58.794657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99abd0e3-1928-48ec-9590-79c4015d5cb9 None admin] RBAC: Adding query filter params (role.id=68d664d01d3340a58359d3fc9d738142, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=ed8a40e4f1034a8e99d0eb505e448db9) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:58.794964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99abd0e3-1928-48ec-9590-79c4015d5cb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:58.796410 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99abd0e3-1928-48ec-9590-79c4015d5cb9 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:58.802769 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 548/1092] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:25:58 2022] GET /identity/v3/role_assignments?user.id=ed8a40e4f1034a8e99d0eb505e448db9&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=68d664d01d3340a58359d3fc9d738142 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:58.816532 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 545/1093] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:25:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 10:25:58.840275 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abd0e3-1928-48ec-9590-79c4015d5cb9 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:58.947707 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abd0e3-1928-48ec-9590-79c4015d5cb9 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:58.983011 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc4fba-1b42-4ffb-92eb-a7005f7767d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:58.983829 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc4fba-1b42-4ffb-92eb-a7005f7767d8 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:58.984112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc4fba-1b42-4ffb-92eb-a7005f7767d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:58.984475 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc4fba-1b42-4ffb-92eb-a7005f7767d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:58.995546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc4fba-1b42-4ffb-92eb-a7005f7767d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:25:59.003711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-d8dc4fba-1b42-4ffb-92eb-a7005f7767d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:25:59.003711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:25:59.003711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:25:59.003711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:25:59.003711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:25:59.003711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:25:59.003711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:25:59.003711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:25:59.003711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:25:59.003711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:25:59.003711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:25:59.003711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:25:59.003711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:25:59.003711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:25:59.003711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:25:59.003711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:25:59.004710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:25:59.004710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:25:59.004710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:25:59.004710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:25:59.004710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:25:59.004710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:25:59.004710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:25:59.004710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:25:59.004710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:25:59.004710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:25:59.004710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:25:59.004710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:25:59.004710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:25:59.004710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:25:59.004710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:25:59.004710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:25:59.005580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:25:59.005580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:25:59.005580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:25:59.005580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:25:59.005580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:25:59.005580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:25:59.005580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:25:59.005580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:25:59.005580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:25:59.005580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:25:59.005580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:25:59.005580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:25:59.005580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 549/1094] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:25:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:25:59.010940 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b94794-3a29-450e-a62f-412a3a2dbbc7 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:59.074153 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:59.114544 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b94794-3a29-450e-a62f-412a3a2dbbc7 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:59.149663 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db7b64-9e42-420b-9b0a-9ff62a2c6780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:59.150325 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db7b64-9e42-420b-9b0a-9ff62a2c6780 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:59.150603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db7b64-9e42-420b-9b0a-9ff62a2c6780 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:59.150874 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db7b64-9e42-420b-9b0a-9ff62a2c6780 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:59.151614 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db7b64-9e42-420b-9b0a-9ff62a2c6780 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:25:59.151950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db7b64-9e42-420b-9b0a-9ff62a2c6780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:59.153305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db7b64-9e42-420b-9b0a-9ff62a2c6780 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:59.163741 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 546/1095] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:25:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:59.169436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc4fba-1b42-4ffb-92eb-a7005f7767d8 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:59.201930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:59.271634 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc4fba-1b42-4ffb-92eb-a7005f7767d8 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:59.308730 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ff122-3beb-4bc8-8912-57f6d05c625e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:59.309603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ff122-3beb-4bc8-8912-57f6d05c625e None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:59.309885 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ff122-3beb-4bc8-8912-57f6d05c625e None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:59.310162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ff122-3beb-4bc8-8912-57f6d05c625e None admin] PATH_INFO: `/v3/users/bde997d1a2004a99890991705714a627` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:59.311350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ff122-3beb-4bc8-8912-57f6d05c625e None admin] RBAC: Authorizing `identity:get_user(user_id=bde997d1a2004a99890991705714a627)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:59.312963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ff122-3beb-4bc8-8912-57f6d05c625e None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:59.314617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 550/1096] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:25:59 2022] GET /identity/v3/users/bde997d1a2004a99890991705714a627 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:59.320588 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db7b64-9e42-420b-9b0a-9ff62a2c6780 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:59.428956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db7b64-9e42-420b-9b0a-9ff62a2c6780 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:59.456715 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:59.467588 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc000c-cee8-4b5d-a832-95db2854d79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:59.468491 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc000c-cee8-4b5d-a832-95db2854d79a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:59.469289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc000c-cee8-4b5d-a832-95db2854d79a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:59.469561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc000c-cee8-4b5d-a832-95db2854d79a None admin] PATH_INFO: `/v3/projects/a67b912ec01746b9ba9657ba9afd5901` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:59.470750 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc000c-cee8-4b5d-a832-95db2854d79a None admin] RBAC: Authorizing `identity:get_project(project_id=a67b912ec01746b9ba9657ba9afd5901)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:59.472241 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc000c-cee8-4b5d-a832-95db2854d79a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:59.473937 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 547/1097] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:25:59 2022] GET /identity/v3/projects/a67b912ec01746b9ba9657ba9afd5901 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:25:59.479881 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ff122-3beb-4bc8-8912-57f6d05c625e None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:25:59.580869 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:25:59.584211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ff122-3beb-4bc8-8912-57f6d05c625e None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:25:59.618661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280791f8-5fc8-4f4c-88ee-106dfa22bbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:25:59.619547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280791f8-5fc8-4f4c-88ee-106dfa22bbc4 None admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:25:59.619849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280791f8-5fc8-4f4c-88ee-106dfa22bbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:25:59.620128 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280791f8-5fc8-4f4c-88ee-106dfa22bbc4 None admin] PATH_INFO: `/v3/projects/a67b912ec01746b9ba9657ba9afd5901/users/bde997d1a2004a99890991705714a627/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:25:59.622082 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280791f8-5fc8-4f4c-88ee-106dfa22bbc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=a67b912ec01746b9ba9657ba9afd5901, user_id=bde997d1a2004a99890991705714a627, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:25:59.623498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280791f8-5fc8-4f4c-88ee-106dfa22bbc4 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:25:59.636479 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 551/1098] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:25:59 2022] PUT /identity/v3/projects/a67b912ec01746b9ba9657ba9afd5901/users/bde997d1a2004a99890991705714a627/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:26:01.871763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64556cf-dfba-4c99-883f-cd249cd54153 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:01.872339 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64556cf-dfba-4c99-883f-cd249cd54153 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:01.872339 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64556cf-dfba-4c99-883f-cd249cd54153 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:01.873672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 548/1099] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:26:01.878981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53395a-6d1d-4b98-b916-db2ddf47aa72 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:01.879419 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53395a-6d1d-4b98-b916-db2ddf47aa72 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:01.879525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53395a-6d1d-4b98-b916-db2ddf47aa72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:01.895980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-9f53395a-6d1d-4b98-b916-db2ddf47aa72 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:26:01.897156 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-9f53395a-6d1d-4b98-b916-db2ddf47aa72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:26:01.962233 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 552/1100] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:26:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:26:01.986500 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64556cf-dfba-4c99-883f-cd249cd54153 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:02.087836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64556cf-dfba-4c99-883f-cd249cd54153 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:02.119478 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728e3fc-38b0-4063-b77f-a2af4609145e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:02.120273 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728e3fc-38b0-4063-b77f-a2af4609145e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:02.120553 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728e3fc-38b0-4063-b77f-a2af4609145e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:02.120818 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728e3fc-38b0-4063-b77f-a2af4609145e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:02.131553 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f728e3fc-38b0-4063-b77f-a2af4609145e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:26:02.139064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-f728e3fc-38b0-4063-b77f-a2af4609145e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:26:02.139064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:26:02.139064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:26:02.139064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:26:02.139064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:26:02.139064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:26:02.139064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:26:02.139064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:26:02.139064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:26:02.139064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:26:02.139064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:26:02.139064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:26:02.139064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:26:02.139064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:26:02.139064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:26:02.139064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:26:02.140161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:26:02.140161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:26:02.140161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:26:02.140161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:26:02.140161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:26:02.140161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:26:02.140161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:26:02.140161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:26:02.140161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:26:02.140161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:26:02.140161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:26:02.140161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:26:02.140161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:26:02.140161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:26:02.140161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:26:02.140161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:26:02.141066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:26:02.141066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:26:02.141066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:26:02.141066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:26:02.141066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:26:02.141066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:26:02.141066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:26:02.141066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:26:02.141066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:26:02.141066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:26:02.141066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:26:02.141066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:26:02.141066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 549/1101] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:26:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:26:02.146158 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53395a-6d1d-4b98-b916-db2ddf47aa72 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:02.237170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:02.247030 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53395a-6d1d-4b98-b916-db2ddf47aa72 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:02.280173 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ba4fe-2aba-48e2-af7b-22d7b9852cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:02.280829 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ba4fe-2aba-48e2-af7b-22d7b9852cae None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:02.281137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ba4fe-2aba-48e2-af7b-22d7b9852cae None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:02.281410 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ba4fe-2aba-48e2-af7b-22d7b9852cae None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:02.282138 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ba4fe-2aba-48e2-af7b-22d7b9852cae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:26:02.282439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ba4fe-2aba-48e2-af7b-22d7b9852cae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:02.283819 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ba4fe-2aba-48e2-af7b-22d7b9852cae None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:02.291681 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 553/1102] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:26:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:02.297979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728e3fc-38b0-4063-b77f-a2af4609145e None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:02.351078 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:02.395249 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728e3fc-38b0-4063-b77f-a2af4609145e None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:02.428333 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d2df6-65c5-4eb7-a65d-00e788569d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:02.429193 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d2df6-65c5-4eb7-a65d-00e788569d56 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:02.429474 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d2df6-65c5-4eb7-a65d-00e788569d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:02.429745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d2df6-65c5-4eb7-a65d-00e788569d56 None admin] PATH_INFO: `/v3/users/bde997d1a2004a99890991705714a627` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:02.430913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d2df6-65c5-4eb7-a65d-00e788569d56 None admin] RBAC: Authorizing `identity:get_user(user_id=bde997d1a2004a99890991705714a627)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:02.432370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d2df6-65c5-4eb7-a65d-00e788569d56 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:02.433968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 550/1103] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:26:02 2022] GET /identity/v3/users/bde997d1a2004a99890991705714a627 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:02.440411 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ba4fe-2aba-48e2-af7b-22d7b9852cae None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:02.537252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ba4fe-2aba-48e2-af7b-22d7b9852cae None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:02.570186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593d171-65f4-4a7d-a6dc-21ced65a4638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:02.570813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593d171-65f4-4a7d-a6dc-21ced65a4638 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:02.571094 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593d171-65f4-4a7d-a6dc-21ced65a4638 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:02.571368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593d171-65f4-4a7d-a6dc-21ced65a4638 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:02.591348 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:02.637193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5593d171-65f4-4a7d-a6dc-21ced65a4638 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:02.638673 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5593d171-65f4-4a7d-a6dc-21ced65a4638 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:02.732537 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 554/1104] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:26:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 10:26:02.738943 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d2df6-65c5-4eb7-a65d-00e788569d56 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:02.835531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d2df6-65c5-4eb7-a65d-00e788569d56 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:02.857348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:02.869568 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f7c7b-9266-4fca-8ff2-2eae649c9347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:02.870413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f7c7b-9266-4fca-8ff2-2eae649c9347 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:02.870689 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f7c7b-9266-4fca-8ff2-2eae649c9347 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:02.870956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f7c7b-9266-4fca-8ff2-2eae649c9347 None admin] PATH_INFO: `/v3/projects/a67b912ec01746b9ba9657ba9afd5901` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:02.872136 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f7c7b-9266-4fca-8ff2-2eae649c9347 None admin] RBAC: Authorizing `identity:get_project(project_id=a67b912ec01746b9ba9657ba9afd5901)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:02.873546 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f7c7b-9266-4fca-8ff2-2eae649c9347 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:02.875496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 551/1105] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:26:02 2022] GET /identity/v3/projects/a67b912ec01746b9ba9657ba9afd5901 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:02.881515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593d171-65f4-4a7d-a6dc-21ced65a4638 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:02.932032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:02.977416 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593d171-65f4-4a7d-a6dc-21ced65a4638 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:03.009350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f67261-597d-4e3d-a016-31c82c4f1a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:03.009986 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f67261-597d-4e3d-a016-31c82c4f1a02 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:03.010268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f67261-597d-4e3d-a016-31c82c4f1a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:03.010546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f67261-597d-4e3d-a016-31c82c4f1a02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:03.011359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f67261-597d-4e3d-a016-31c82c4f1a02 None admin] RBAC: Adding query filter params (role.id=0f929204eb7b4ea8859338ed2d3ff75b, scope.project.id=a67b912ec01746b9ba9657ba9afd5901, user.id=bde997d1a2004a99890991705714a627) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:26:03.011665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f67261-597d-4e3d-a016-31c82c4f1a02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:03.013088 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f67261-597d-4e3d-a016-31c82c4f1a02 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:03.019335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 555/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:26:02 2022] GET /identity/v3/role_assignments?user.id=bde997d1a2004a99890991705714a627&scope.project.id=a67b912ec01746b9ba9657ba9afd5901&role.id=0f929204eb7b4ea8859338ed2d3ff75b => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:05.260535 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9b5be-738b-4bb1-8916-e6d1609de223 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:05.261066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9b5be-738b-4bb1-8916-e6d1609de223 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:05.261066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9b5be-738b-4bb1-8916-e6d1609de223 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:05.262416 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 552/1107] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:26:05.267914 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda7a36-1d45-40ea-b1bc-ca5bb1cbf293 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:05.268333 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda7a36-1d45-40ea-b1bc-ca5bb1cbf293 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:05.268475 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda7a36-1d45-40ea-b1bc-ca5bb1cbf293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:05.285063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-ffda7a36-1d45-40ea-b1bc-ca5bb1cbf293 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:26:05.286241 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ffda7a36-1d45-40ea-b1bc-ca5bb1cbf293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:26:05.352777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 556/1108] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:26:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:26:05.378399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9b5be-738b-4bb1-8916-e6d1609de223 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:05.476754 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9b5be-738b-4bb1-8916-e6d1609de223 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:05.509288 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06259257-488a-4103-b7ca-e77df2e5734c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:05.510071 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06259257-488a-4103-b7ca-e77df2e5734c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:05.510351 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06259257-488a-4103-b7ca-e77df2e5734c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:05.510676 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06259257-488a-4103-b7ca-e77df2e5734c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:05.511808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06259257-488a-4103-b7ca-e77df2e5734c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:05.513238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06259257-488a-4103-b7ca-e77df2e5734c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:05.514828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 553/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:26:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:05.520740 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda7a36-1d45-40ea-b1bc-ca5bb1cbf293 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:05.566882 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:05.617546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda7a36-1d45-40ea-b1bc-ca5bb1cbf293 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:05.650817 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e365c-4de1-4de0-9ae5-65b1b1308965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:05.651465 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e365c-4de1-4de0-9ae5-65b1b1308965 None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:05.651747 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e365c-4de1-4de0-9ae5-65b1b1308965 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:05.652051 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e365c-4de1-4de0-9ae5-65b1b1308965 None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:05.652754 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e365c-4de1-4de0-9ae5-65b1b1308965 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:05.654176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e365c-4de1-4de0-9ae5-65b1b1308965 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:05.678055 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 557/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jun 6 10:26:05 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:26:07.955381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070a6e0-b508-497b-af85-2babd05d06bd None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:07.956009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070a6e0-b508-497b-af85-2babd05d06bd None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:07.956009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070a6e0-b508-497b-af85-2babd05d06bd None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:07.957337 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 554/1111] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:26:07.962602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1dd68-a88b-4c92-a1ce-568b1de8caee None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:07.962995 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1dd68-a88b-4c92-a1ce-568b1de8caee None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:07.963156 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1dd68-a88b-4c92-a1ce-568b1de8caee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:07.980004 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-25a1dd68-a88b-4c92-a1ce-568b1de8caee None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:26:07.981189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-25a1dd68-a88b-4c92-a1ce-568b1de8caee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:26:08.045530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 558/1112] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:26:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:26:08.069252 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070a6e0-b508-497b-af85-2babd05d06bd None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:08.126864 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:08.167829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070a6e0-b508-497b-af85-2babd05d06bd None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:08.200772 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1bb1f-3907-4ebb-a037-cde04da66cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:08.201600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1bb1f-3907-4ebb-a037-cde04da66cee None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:08.201878 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1bb1f-3907-4ebb-a037-cde04da66cee None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:08.202146 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1bb1f-3907-4ebb-a037-cde04da66cee None admin] PATH_INFO: `/v3/roles/f214f8dbc3354cfc81fa2c2f116980e0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:08.204211 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1bb1f-3907-4ebb-a037-cde04da66cee None admin] RBAC: Authorizing `identity:get_role(role_id=f214f8dbc3354cfc81fa2c2f116980e0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:08.205615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1bb1f-3907-4ebb-a037-cde04da66cee None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:08.206890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 555/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:26:08 2022] GET /identity/v3/roles/f214f8dbc3354cfc81fa2c2f116980e0 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:08.212558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1dd68-a88b-4c92-a1ce-568b1de8caee None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:08.242269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:08.309541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1dd68-a88b-4c92-a1ce-568b1de8caee None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:08.342052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f8017-b029-4dab-9340-646869d6e02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:08.342903 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f8017-b029-4dab-9340-646869d6e02a None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:08.343204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f8017-b029-4dab-9340-646869d6e02a None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:08.344037 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f8017-b029-4dab-9340-646869d6e02a None admin] PATH_INFO: `/v3/users/d5a1fecc503443c0aa6385fc6fafd4d8` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:08.356566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f8017-b029-4dab-9340-646869d6e02a None admin] RBAC: Authorizing `identity:get_user(user_id=d5a1fecc503443c0aa6385fc6fafd4d8)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:08.358001 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f8017-b029-4dab-9340-646869d6e02a None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:08.359645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 559/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:26:08 2022] GET /identity/v3/users/d5a1fecc503443c0aa6385fc6fafd4d8 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:08.365315 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1bb1f-3907-4ebb-a037-cde04da66cee None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:08.461117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1bb1f-3907-4ebb-a037-cde04da66cee None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:08.485564 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:08.494273 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404550a-2737-46eb-8b80-74e347f9b970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:08.494915 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404550a-2737-46eb-8b80-74e347f9b970 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:08.495199 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404550a-2737-46eb-8b80-74e347f9b970 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:08.495469 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404550a-2737-46eb-8b80-74e347f9b970 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:08.560409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404550a-2737-46eb-8b80-74e347f9b970 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:08.561842 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404550a-2737-46eb-8b80-74e347f9b970 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:08.655498 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 556/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:26:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 10:26:08.661551 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f8017-b029-4dab-9340-646869d6e02a None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:08.747835 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:08.758094 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f8017-b029-4dab-9340-646869d6e02a None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:08.794758 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849532a-a9ce-4021-a36b-5e16c8961bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:08.795624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849532a-a9ce-4021-a36b-5e16c8961bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:08.795939 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849532a-a9ce-4021-a36b-5e16c8961bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:08.796215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849532a-a9ce-4021-a36b-5e16c8961bc6 None admin] PATH_INFO: `/v3/projects/a67b912ec01746b9ba9657ba9afd5901` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:08.798052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3849532a-a9ce-4021-a36b-5e16c8961bc6 None admin] RBAC: Authorizing `identity:get_project(project_id=a67b912ec01746b9ba9657ba9afd5901)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:08.799476 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3849532a-a9ce-4021-a36b-5e16c8961bc6 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:08.801209 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 560/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:26:08 2022] GET /identity/v3/projects/a67b912ec01746b9ba9657ba9afd5901 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:08.807406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404550a-2737-46eb-8b80-74e347f9b970 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:08.833319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:08.904192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404550a-2737-46eb-8b80-74e347f9b970 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:08.936488 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b90103-a58d-4a36-95bb-d153f6a1a44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:08.937131 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b90103-a58d-4a36-95bb-d153f6a1a44c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:08.937415 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b90103-a58d-4a36-95bb-d153f6a1a44c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:08.937682 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b90103-a58d-4a36-95bb-d153f6a1a44c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:08.938498 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b90103-a58d-4a36-95bb-d153f6a1a44c None admin] RBAC: Adding query filter params (role.id=f214f8dbc3354cfc81fa2c2f116980e0, scope.project.id=a67b912ec01746b9ba9657ba9afd5901, user.id=d5a1fecc503443c0aa6385fc6fafd4d8) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:26:08.938804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b90103-a58d-4a36-95bb-d153f6a1a44c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:08.940257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b90103-a58d-4a36-95bb-d153f6a1a44c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:08.945490 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 557/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:26:08 2022] GET /identity/v3/role_assignments?user.id=d5a1fecc503443c0aa6385fc6fafd4d8&scope.project.id=a67b912ec01746b9ba9657ba9afd5901&role.id=f214f8dbc3354cfc81fa2c2f116980e0 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:11.153033 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded88dff-40af-4277-8ae1-a29f65023d0d None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:11.153558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded88dff-40af-4277-8ae1-a29f65023d0d None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:11.153558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded88dff-40af-4277-8ae1-a29f65023d0d None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:11.154925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 561/1118] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:26:11.160697 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ffd42-0996-4a70-8706-cd78f7e42fcb None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:11.161111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ffd42-0996-4a70-8706-cd78f7e42fcb None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:11.161251 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ffd42-0996-4a70-8706-cd78f7e42fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:11.177621 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3a8ffd42-0996-4a70-8706-cd78f7e42fcb None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:26:11.178791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3a8ffd42-0996-4a70-8706-cd78f7e42fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:26:11.243120 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 558/1119] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:26:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:26:11.267151 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded88dff-40af-4277-8ae1-a29f65023d0d None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:11.366800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded88dff-40af-4277-8ae1-a29f65023d0d None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:11.400530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33eb784-ae99-4a66-ae67-5d5cc4bf11c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:11.401382 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33eb784-ae99-4a66-ae67-5d5cc4bf11c6 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:11.401665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33eb784-ae99-4a66-ae67-5d5cc4bf11c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:11.401941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33eb784-ae99-4a66-ae67-5d5cc4bf11c6 None admin] PATH_INFO: `/v3/roles/f214f8dbc3354cfc81fa2c2f116980e0` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:11.403431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33eb784-ae99-4a66-ae67-5d5cc4bf11c6 None admin] RBAC: Authorizing `identity:get_role(role_id=f214f8dbc3354cfc81fa2c2f116980e0)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:11.404777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33eb784-ae99-4a66-ae67-5d5cc4bf11c6 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:11.406693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 562/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:26:11 2022] GET /identity/v3/roles/f214f8dbc3354cfc81fa2c2f116980e0 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:11.412571 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ffd42-0996-4a70-8706-cd78f7e42fcb None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:11.437458 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:11.513543 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ffd42-0996-4a70-8706-cd78f7e42fcb None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:11.546225 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055d84f-08fb-4b0f-9c37-b56ff53b193f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:11.547167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055d84f-08fb-4b0f-9c37-b56ff53b193f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:11.547449 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055d84f-08fb-4b0f-9c37-b56ff53b193f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:11.547720 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055d84f-08fb-4b0f-9c37-b56ff53b193f None admin] PATH_INFO: `/v3/users/d5a1fecc503443c0aa6385fc6fafd4d8` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:11.548942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055d84f-08fb-4b0f-9c37-b56ff53b193f None admin] RBAC: Authorizing `identity:get_user(user_id=d5a1fecc503443c0aa6385fc6fafd4d8)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:11.550370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055d84f-08fb-4b0f-9c37-b56ff53b193f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:11.552001 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 559/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:26:11 2022] GET /identity/v3/users/d5a1fecc503443c0aa6385fc6fafd4d8 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:11.557849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33eb784-ae99-4a66-ae67-5d5cc4bf11c6 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:11.585524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:11.656111 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33eb784-ae99-4a66-ae67-5d5cc4bf11c6 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:11.689431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90078baf-06c6-40b0-8afc-aadc52edad56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:11.690274 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90078baf-06c6-40b0-8afc-aadc52edad56 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:11.690583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90078baf-06c6-40b0-8afc-aadc52edad56 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:11.690854 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90078baf-06c6-40b0-8afc-aadc52edad56 None admin] PATH_INFO: `/v3/projects/a67b912ec01746b9ba9657ba9afd5901` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:11.692027 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90078baf-06c6-40b0-8afc-aadc52edad56 None admin] RBAC: Authorizing `identity:get_project(project_id=a67b912ec01746b9ba9657ba9afd5901)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:11.693450 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90078baf-06c6-40b0-8afc-aadc52edad56 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:11.695110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 563/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:26:11 2022] GET /identity/v3/projects/a67b912ec01746b9ba9657ba9afd5901 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:11.701013 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055d84f-08fb-4b0f-9c37-b56ff53b193f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:11.793873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:11.806674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055d84f-08fb-4b0f-9c37-b56ff53b193f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:11.838978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5544e-e12a-4a30-9419-fdd4f1571a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:11.839879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5544e-e12a-4a30-9419-fdd4f1571a2a None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:11.840161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5544e-e12a-4a30-9419-fdd4f1571a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:11.840432 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5544e-e12a-4a30-9419-fdd4f1571a2a None admin] PATH_INFO: `/v3/projects/a67b912ec01746b9ba9657ba9afd5901/users/d5a1fecc503443c0aa6385fc6fafd4d8/roles/f214f8dbc3354cfc81fa2c2f116980e0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:11.842569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5544e-e12a-4a30-9419-fdd4f1571a2a None admin] RBAC: Authorizing `identity:create_grant(project_id=a67b912ec01746b9ba9657ba9afd5901, user_id=d5a1fecc503443c0aa6385fc6fafd4d8, role_id=f214f8dbc3354cfc81fa2c2f116980e0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:11.844842 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5544e-e12a-4a30-9419-fdd4f1571a2a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:11.856753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 560/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:26:11 2022] PUT /identity/v3/projects/a67b912ec01746b9ba9657ba9afd5901/users/d5a1fecc503443c0aa6385fc6fafd4d8/roles/f214f8dbc3354cfc81fa2c2f116980e0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:26:14.053379 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b365c7a-d4d7-48b2-8db8-cae796c33d1b None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:14.053935 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b365c7a-d4d7-48b2-8db8-cae796c33d1b None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:14.053935 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b365c7a-d4d7-48b2-8db8-cae796c33d1b None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:14.055286 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 564/1124] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:26:14.060613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9644974-3bbe-490c-8f0e-160cb164b8d6 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:14.060996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9644974-3bbe-490c-8f0e-160cb164b8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:14.061166 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9644974-3bbe-490c-8f0e-160cb164b8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:14.077392 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b9644974-3bbe-490c-8f0e-160cb164b8d6 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:26:14.078550 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b9644974-3bbe-490c-8f0e-160cb164b8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:26:14.143087 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 561/1125] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:26:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:26:14.167312 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b365c7a-d4d7-48b2-8db8-cae796c33d1b None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:14.258343 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:14.266568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b365c7a-d4d7-48b2-8db8-cae796c33d1b None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:14.299527 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5898f3-33e6-4418-a714-177dda087548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:14.300384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5898f3-33e6-4418-a714-177dda087548 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:14.300664 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5898f3-33e6-4418-a714-177dda087548 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:14.300937 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5898f3-33e6-4418-a714-177dda087548 None admin] PATH_INFO: `/v3/roles/f214f8dbc3354cfc81fa2c2f116980e0` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:14.302374 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5898f3-33e6-4418-a714-177dda087548 None admin] RBAC: Authorizing `identity:get_role(role_id=f214f8dbc3354cfc81fa2c2f116980e0)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:14.303766 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5898f3-33e6-4418-a714-177dda087548 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:14.305097 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 565/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:26:14 2022] GET /identity/v3/roles/f214f8dbc3354cfc81fa2c2f116980e0 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:14.310779 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9644974-3bbe-490c-8f0e-160cb164b8d6 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:14.384783 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:14.408159 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9644974-3bbe-490c-8f0e-160cb164b8d6 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:14.440561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d3f71-bc6d-48d4-9900-b0f87090e0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:14.441419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d3f71-bc6d-48d4-9900-b0f87090e0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:14.441700 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d3f71-bc6d-48d4-9900-b0f87090e0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:14.441971 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d3f71-bc6d-48d4-9900-b0f87090e0e1 None admin] PATH_INFO: `/v3/users/d5a1fecc503443c0aa6385fc6fafd4d8` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:14.443148 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d3f71-bc6d-48d4-9900-b0f87090e0e1 None admin] RBAC: Authorizing `identity:get_user(user_id=d5a1fecc503443c0aa6385fc6fafd4d8)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:14.444606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d3f71-bc6d-48d4-9900-b0f87090e0e1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:14.446217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 562/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:26:14 2022] GET /identity/v3/users/d5a1fecc503443c0aa6385fc6fafd4d8 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:14.451995 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5898f3-33e6-4418-a714-177dda087548 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:14.548598 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5898f3-33e6-4418-a714-177dda087548 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:14.581852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759eaec3-bf7f-410a-88cd-9ae269ff51a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:14.582415 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759eaec3-bf7f-410a-88cd-9ae269ff51a3 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:14.582697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759eaec3-bf7f-410a-88cd-9ae269ff51a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:14.582968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759eaec3-bf7f-410a-88cd-9ae269ff51a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:14.611177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:14.648593 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759eaec3-bf7f-410a-88cd-9ae269ff51a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:14.650022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759eaec3-bf7f-410a-88cd-9ae269ff51a3 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:14.743651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 566/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:26:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 10:26:14.750102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d3f71-bc6d-48d4-9900-b0f87090e0e1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:14.847238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d3f71-bc6d-48d4-9900-b0f87090e0e1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:14.879892 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20af337-7527-46db-993b-c854b068348a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:14.880735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20af337-7527-46db-993b-c854b068348a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:14.881020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20af337-7527-46db-993b-c854b068348a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:14.881294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20af337-7527-46db-993b-c854b068348a None admin] PATH_INFO: `/v3/projects/a67b912ec01746b9ba9657ba9afd5901` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:14.882460 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20af337-7527-46db-993b-c854b068348a None admin] RBAC: Authorizing `identity:get_project(project_id=a67b912ec01746b9ba9657ba9afd5901)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:14.883915 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20af337-7527-46db-993b-c854b068348a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:14.885566 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 563/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:26:14 2022] GET /identity/v3/projects/a67b912ec01746b9ba9657ba9afd5901 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:14.892489 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759eaec3-bf7f-410a-88cd-9ae269ff51a3 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:14.951006 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:14.989434 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759eaec3-bf7f-410a-88cd-9ae269ff51a3 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:15.022075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6102988-4f65-4317-94ca-67c2a2c1bd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:15.022974 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6102988-4f65-4317-94ca-67c2a2c1bd55 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:15.023261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6102988-4f65-4317-94ca-67c2a2c1bd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:15.023537 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6102988-4f65-4317-94ca-67c2a2c1bd55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:15.024384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6102988-4f65-4317-94ca-67c2a2c1bd55 None admin] RBAC: Adding query filter params (role.id=f214f8dbc3354cfc81fa2c2f116980e0, scope.project.id=a67b912ec01746b9ba9657ba9afd5901, user.id=d5a1fecc503443c0aa6385fc6fafd4d8) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:26:15.024696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6102988-4f65-4317-94ca-67c2a2c1bd55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:15.026107 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6102988-4f65-4317-94ca-67c2a2c1bd55 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:15.032350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 567/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:26:14 2022] GET /identity/v3/role_assignments?user.id=d5a1fecc503443c0aa6385fc6fafd4d8&scope.project.id=a67b912ec01746b9ba9657ba9afd5901&role.id=f214f8dbc3354cfc81fa2c2f116980e0 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:17.502288 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc74363-91df-4a66-a0a0-9b85d10a785b None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:17.502839 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc74363-91df-4a66-a0a0-9b85d10a785b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:17.502839 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc74363-91df-4a66-a0a0-9b85d10a785b None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:17.504225 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 564/1131] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:26:17.509512 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ddcfd-ddaf-4b99-8ba1-2414c1cbd313 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:17.509889 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ddcfd-ddaf-4b99-8ba1-2414c1cbd313 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:17.510061 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ddcfd-ddaf-4b99-8ba1-2414c1cbd313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:17.527042 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-439ddcfd-ddaf-4b99-8ba1-2414c1cbd313 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:26:17.528237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-439ddcfd-ddaf-4b99-8ba1-2414c1cbd313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:26:17.601036 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 568/1132] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:26:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:26:17.626144 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc74363-91df-4a66-a0a0-9b85d10a785b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:17.642134 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:17.724841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc74363-91df-4a66-a0a0-9b85d10a785b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:17.757100 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe2ed1-36b1-4908-a523-c762079870a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:17.757875 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe2ed1-36b1-4908-a523-c762079870a4 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:17.758154 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe2ed1-36b1-4908-a523-c762079870a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:17.758426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe2ed1-36b1-4908-a523-c762079870a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:17.759576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe2ed1-36b1-4908-a523-c762079870a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:17.761033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe2ed1-36b1-4908-a523-c762079870a4 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:17.762617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 565/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:26:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:17.768956 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ddcfd-ddaf-4b99-8ba1-2414c1cbd313 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:17.823492 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:17.865739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ddcfd-ddaf-4b99-8ba1-2414c1cbd313 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:17.902385 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbad0294-31fc-4d18-8151-3d9e42b41c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:17.903045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad0294-31fc-4d18-8151-3d9e42b41c82 None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:17.903331 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad0294-31fc-4d18-8151-3d9e42b41c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:17.903605 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad0294-31fc-4d18-8151-3d9e42b41c82 None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:17.904346 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbad0294-31fc-4d18-8151-3d9e42b41c82 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:17.906062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbad0294-31fc-4d18-8151-3d9e42b41c82 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:17.931883 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 569/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jun 6 10:26:17 2022] POST /identity/v3/projects => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:26:20.180602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed5577-b645-47a9-bfe8-8b5b10748824 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:20.181154 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed5577-b645-47a9-bfe8-8b5b10748824 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:20.181154 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed5577-b645-47a9-bfe8-8b5b10748824 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:20.182505 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 566/1135] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:26:20.187938 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22d438-3bde-4625-afbf-0652d36870fb None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:20.188375 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22d438-3bde-4625-afbf-0652d36870fb None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:20.188500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22d438-3bde-4625-afbf-0652d36870fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:20.205483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-bb22d438-3bde-4625-afbf-0652d36870fb None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:26:20.206658 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-bb22d438-3bde-4625-afbf-0652d36870fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:26:20.271412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 570/1136] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:26:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 10:26:20.297832 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed5577-b645-47a9-bfe8-8b5b10748824 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:20.379021 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:20.399442 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed5577-b645-47a9-bfe8-8b5b10748824 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:20.432524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92ec20-7c69-4969-8118-190086e2ffbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:20.433242 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92ec20-7c69-4969-8118-190086e2ffbc None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:20.433522 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92ec20-7c69-4969-8118-190086e2ffbc None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:20.433793 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92ec20-7c69-4969-8118-190086e2ffbc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:20.434943 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92ec20-7c69-4969-8118-190086e2ffbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:20.436479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92ec20-7c69-4969-8118-190086e2ffbc None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:20.438236 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 567/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:26:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:20.444061 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22d438-3bde-4625-afbf-0652d36870fb None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:20.482562 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:20.540764 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22d438-3bde-4625-afbf-0652d36870fb None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:20.573203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07afde-f064-4984-89f5-8c1f7886439c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:20.573857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07afde-f064-4984-89f5-8c1f7886439c None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:20.574139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07afde-f064-4984-89f5-8c1f7886439c None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:20.574421 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07afde-f064-4984-89f5-8c1f7886439c None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:20.575127 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d07afde-f064-4984-89f5-8c1f7886439c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:20.576567 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d07afde-f064-4984-89f5-8c1f7886439c None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:20.595750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 571/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jun 6 10:26:20 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:26:22.866136 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09583af6-5623-4739-b738-e1f21df17088 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:22.866690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09583af6-5623-4739-b738-e1f21df17088 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:22.866690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09583af6-5623-4739-b738-e1f21df17088 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:22.868051 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 568/1139] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:26:22.873182 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29847cc-3406-4102-8168-c3d9ef08f8bc None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:22.873556 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29847cc-3406-4102-8168-c3d9ef08f8bc None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:22.874335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29847cc-3406-4102-8168-c3d9ef08f8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:22.891076 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-d29847cc-3406-4102-8168-c3d9ef08f8bc None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:26:22.892270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-d29847cc-3406-4102-8168-c3d9ef08f8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:26:22.957001 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 572/1140] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:26:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:26:22.980597 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09583af6-5623-4739-b738-e1f21df17088 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:23.080805 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09583af6-5623-4739-b738-e1f21df17088 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:23.113616 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb47dd-ed90-4179-9056-9aa4a023cb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:23.114390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb47dd-ed90-4179-9056-9aa4a023cb2c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:23.114673 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb47dd-ed90-4179-9056-9aa4a023cb2c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:23.114947 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb47dd-ed90-4179-9056-9aa4a023cb2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:23.125316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb47dd-ed90-4179-9056-9aa4a023cb2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:26:23.132995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-08bb47dd-ed90-4179-9056-9aa4a023cb2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:26:23.132995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:26:23.132995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:26:23.132995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:26:23.132995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:26:23.132995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:26:23.132995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:26:23.132995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:26:23.132995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:26:23.132995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:26:23.132995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:26:23.132995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:26:23.132995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:26:23.132995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:26:23.132995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:26:23.132995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:26:23.134079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:26:23.134079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:26:23.134079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:26:23.134079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:26:23.134079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:26:23.134079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:26:23.134079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:26:23.134079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:26:23.134079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:26:23.134079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:26:23.134079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:26:23.134079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:26:23.134079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:26:23.134079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:26:23.134079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:26:23.134079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:26:23.135259 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:26:23.135259 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:26:23.135259 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:26:23.135259 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:26:23.135259 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:26:23.135259 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:26:23.135259 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:26:23.135259 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:26:23.135259 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:26:23.135259 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:26:23.135259 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:26:23.135259 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:26:23.135259 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 569/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:26:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:26:23.140139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29847cc-3406-4102-8168-c3d9ef08f8bc None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:23.164001 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:23.237010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29847cc-3406-4102-8168-c3d9ef08f8bc None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:23.270276 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca8844-90a5-4bae-bd6e-a1f315a3aae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:23.270932 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca8844-90a5-4bae-bd6e-a1f315a3aae5 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:23.271219 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca8844-90a5-4bae-bd6e-a1f315a3aae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:23.271498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca8844-90a5-4bae-bd6e-a1f315a3aae5 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:23.272527 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca8844-90a5-4bae-bd6e-a1f315a3aae5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:26:23.272836 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca8844-90a5-4bae-bd6e-a1f315a3aae5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:23.274192 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca8844-90a5-4bae-bd6e-a1f315a3aae5 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:23.282130 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 573/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:26:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:23.287792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb47dd-ed90-4179-9056-9aa4a023cb2c None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:23.300891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:23.390129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb47dd-ed90-4179-9056-9aa4a023cb2c None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:23.423188 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d176c-f1d9-48aa-84bd-6376fd9af39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:23.424082 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d176c-f1d9-48aa-84bd-6376fd9af39f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:23.424375 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d176c-f1d9-48aa-84bd-6376fd9af39f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:23.424651 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d176c-f1d9-48aa-84bd-6376fd9af39f None admin] PATH_INFO: `/v3/users/d118e5dd9188403ca1aeb553cbcaa9c5` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:23.438265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d176c-f1d9-48aa-84bd-6376fd9af39f None admin] RBAC: Authorizing `identity:get_user(user_id=d118e5dd9188403ca1aeb553cbcaa9c5)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:23.439692 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d176c-f1d9-48aa-84bd-6376fd9af39f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:23.441355 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 570/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:26:23 2022] GET /identity/v3/users/d118e5dd9188403ca1aeb553cbcaa9c5 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:23.447065 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca8844-90a5-4bae-bd6e-a1f315a3aae5 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:23.531398 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:23.546577 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca8844-90a5-4bae-bd6e-a1f315a3aae5 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:23.578916 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0923fbe-4b4a-45d6-af79-88d6d7b68425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:23.579546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0923fbe-4b4a-45d6-af79-88d6d7b68425 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:23.579853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0923fbe-4b4a-45d6-af79-88d6d7b68425 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:23.580129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0923fbe-4b4a-45d6-af79-88d6d7b68425 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:23.645992 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0923fbe-4b4a-45d6-af79-88d6d7b68425 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:23.647666 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0923fbe-4b4a-45d6-af79-88d6d7b68425 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:23.726752 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:23.742525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 574/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:26:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 10:26:23.748341 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d176c-f1d9-48aa-84bd-6376fd9af39f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:23.813128 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:23.845850 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d176c-f1d9-48aa-84bd-6376fd9af39f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:23.879409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b11d3-9516-4278-a997-3afa425513a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:23.880294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b11d3-9516-4278-a997-3afa425513a6 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:23.880584 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b11d3-9516-4278-a997-3afa425513a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:23.880858 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b11d3-9516-4278-a997-3afa425513a6 None admin] PATH_INFO: `/v3/projects/972cb8083d0c4f3397d89628d338a1d9` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:23.882023 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b11d3-9516-4278-a997-3afa425513a6 None admin] RBAC: Authorizing `identity:get_project(project_id=972cb8083d0c4f3397d89628d338a1d9)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:23.883447 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b11d3-9516-4278-a997-3afa425513a6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:23.885156 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 571/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:26:23 2022] GET /identity/v3/projects/972cb8083d0c4f3397d89628d338a1d9 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:23.891300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0923fbe-4b4a-45d6-af79-88d6d7b68425 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:23.987190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0923fbe-4b4a-45d6-af79-88d6d7b68425 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:24.020157 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec25564-c40a-4ca4-ae53-f6ef46ef3414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:24.020780 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec25564-c40a-4ca4-ae53-f6ef46ef3414 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:24.021063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec25564-c40a-4ca4-ae53-f6ef46ef3414 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:24.021334 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec25564-c40a-4ca4-ae53-f6ef46ef3414 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:24.022399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec25564-c40a-4ca4-ae53-f6ef46ef3414 None admin] RBAC: Adding query filter params (role.id=0f929204eb7b4ea8859338ed2d3ff75b, scope.project.id=972cb8083d0c4f3397d89628d338a1d9, user.id=d118e5dd9188403ca1aeb553cbcaa9c5) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:26:24.022700 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec25564-c40a-4ca4-ae53-f6ef46ef3414 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:24.024111 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec25564-c40a-4ca4-ae53-f6ef46ef3414 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:24.029326 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 575/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:26:23 2022] GET /identity/v3/role_assignments?user.id=d118e5dd9188403ca1aeb553cbcaa9c5&scope.project.id=972cb8083d0c4f3397d89628d338a1d9&role.id=0f929204eb7b4ea8859338ed2d3ff75b => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:26.241792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7aea3-ba6f-4d09-b66b-ab5afd9bf59f None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:26.242345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7aea3-ba6f-4d09-b66b-ab5afd9bf59f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:26.242345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7aea3-ba6f-4d09-b66b-ab5afd9bf59f None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:26.243822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 572/1147] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:26:26.248981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f6c9a-ad85-4329-8046-394356bfd384 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:26.249399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f6c9a-ad85-4329-8046-394356bfd384 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:26.249672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f6c9a-ad85-4329-8046-394356bfd384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:26.266451 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-f62f6c9a-ad85-4329-8046-394356bfd384 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:26:26.267626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-f62f6c9a-ad85-4329-8046-394356bfd384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:26:26.313223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:26.333887 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 576/1148] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:26:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:26:26.356062 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7aea3-ba6f-4d09-b66b-ab5afd9bf59f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:26.457408 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7aea3-ba6f-4d09-b66b-ab5afd9bf59f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:26.490332 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab4f9c-1fc7-4c8a-9be9-f3417b164149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:26.491109 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab4f9c-1fc7-4c8a-9be9-f3417b164149 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:26.491387 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab4f9c-1fc7-4c8a-9be9-f3417b164149 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:26.491659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab4f9c-1fc7-4c8a-9be9-f3417b164149 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:26.494283 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:26.502569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddab4f9c-1fc7-4c8a-9be9-f3417b164149 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:26:26.510192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-ddab4f9c-1fc7-4c8a-9be9-f3417b164149 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:26:26.510192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:26:26.510192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:26:26.510192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:26:26.510192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:26:26.510192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:26:26.510192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:26:26.510192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:26:26.510192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:26:26.510192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:26:26.510192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:26:26.510192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:26:26.510192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:26:26.510192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:26:26.510192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:26:26.510192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:26:26.511257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:26:26.511257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:26:26.511257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:26:26.511257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:26:26.511257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:26:26.511257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:26:26.511257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:26:26.511257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:26:26.511257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:26:26.511257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:26:26.511257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:26:26.511257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:26:26.511257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:26:26.511257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:26:26.511257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:26:26.511257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:26:26.512314 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:26:26.512314 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:26:26.512314 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:26:26.512314 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:26:26.512314 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:26:26.512314 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:26:26.512314 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:26:26.512314 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:26:26.512314 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:26:26.512314 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:26:26.512314 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:26:26.512314 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:26:26.512314 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 573/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:26:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:26:26.517445 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f6c9a-ad85-4329-8046-394356bfd384 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:26.613899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f6c9a-ad85-4329-8046-394356bfd384 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:26.646808 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15172875-2a54-491a-8248-156be8a64234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:26.647705 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15172875-2a54-491a-8248-156be8a64234 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:26.648008 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15172875-2a54-491a-8248-156be8a64234 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:26.648280 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15172875-2a54-491a-8248-156be8a64234 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:26.649007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15172875-2a54-491a-8248-156be8a64234 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:26:26.649307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15172875-2a54-491a-8248-156be8a64234 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:26.650643 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15172875-2a54-491a-8248-156be8a64234 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:26.659155 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 577/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:26:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:26.665024 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab4f9c-1fc7-4c8a-9be9-f3417b164149 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:26.760806 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab4f9c-1fc7-4c8a-9be9-f3417b164149 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:26.793045 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136d70a-a8a9-45a3-805f-5fcb81126277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:26.793893 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136d70a-a8a9-45a3-805f-5fcb81126277 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:26.794174 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136d70a-a8a9-45a3-805f-5fcb81126277 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:26.794447 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136d70a-a8a9-45a3-805f-5fcb81126277 None admin] PATH_INFO: `/v3/users/d118e5dd9188403ca1aeb553cbcaa9c5` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:26.796177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e136d70a-a8a9-45a3-805f-5fcb81126277 None admin] RBAC: Authorizing `identity:get_user(user_id=d118e5dd9188403ca1aeb553cbcaa9c5)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:26.797584 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e136d70a-a8a9-45a3-805f-5fcb81126277 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:26.799166 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 574/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:26:26 2022] GET /identity/v3/users/d118e5dd9188403ca1aeb553cbcaa9c5 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:26.804874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15172875-2a54-491a-8248-156be8a64234 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:26.846700 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:26.902924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15172875-2a54-491a-8248-156be8a64234 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:26.940406 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5869e8-6ddb-4031-be4c-1f5008ebdab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:26.941260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5869e8-6ddb-4031-be4c-1f5008ebdab4 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:26.941543 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5869e8-6ddb-4031-be4c-1f5008ebdab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:26.941815 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5869e8-6ddb-4031-be4c-1f5008ebdab4 None admin] PATH_INFO: `/v3/projects/972cb8083d0c4f3397d89628d338a1d9` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:26.942985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5869e8-6ddb-4031-be4c-1f5008ebdab4 None admin] RBAC: Authorizing `identity:get_project(project_id=972cb8083d0c4f3397d89628d338a1d9)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:26.944444 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5869e8-6ddb-4031-be4c-1f5008ebdab4 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:26.946138 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 578/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:26:26 2022] GET /identity/v3/projects/972cb8083d0c4f3397d89628d338a1d9 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:26.952611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136d70a-a8a9-45a3-805f-5fcb81126277 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:27.012918 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:27.053879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136d70a-a8a9-45a3-805f-5fcb81126277 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:27.086109 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08526a48-6c34-42da-9e5c-70a00655a758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:27.086979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08526a48-6c34-42da-9e5c-70a00655a758 None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:27.087253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08526a48-6c34-42da-9e5c-70a00655a758 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:27.087520 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08526a48-6c34-42da-9e5c-70a00655a758 None admin] PATH_INFO: `/v3/projects/972cb8083d0c4f3397d89628d338a1d9/users/d118e5dd9188403ca1aeb553cbcaa9c5/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:27.089473 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08526a48-6c34-42da-9e5c-70a00655a758 None admin] RBAC: Authorizing `identity:create_grant(project_id=972cb8083d0c4f3397d89628d338a1d9, user_id=d118e5dd9188403ca1aeb553cbcaa9c5, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:27.090890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08526a48-6c34-42da-9e5c-70a00655a758 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:27.103571 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 575/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:26:26 2022] PUT /identity/v3/projects/972cb8083d0c4f3397d89628d338a1d9/users/d118e5dd9188403ca1aeb553cbcaa9c5/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:26:29.381952 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31ad19-d290-452a-8409-e5d34da15094 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:29.382428 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31ad19-d290-452a-8409-e5d34da15094 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:29.382526 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31ad19-d290-452a-8409-e5d34da15094 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:29.383874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 579/1154] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:26:29.390261 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5948c-8e3e-4f29-9b68-ca8a8be043c6 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:29.390718 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5948c-8e3e-4f29-9b68-ca8a8be043c6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:29.390821 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5948c-8e3e-4f29-9b68-ca8a8be043c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:29.408480 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-bad5948c-8e3e-4f29-9b68-ca8a8be043c6 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:26:29.409650 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bad5948c-8e3e-4f29-9b68-ca8a8be043c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:26:29.475084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 576/1155] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:26:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:26:29.499244 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31ad19-d290-452a-8409-e5d34da15094 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:29.599576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31ad19-d290-452a-8409-e5d34da15094 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:29.610150 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:29.632785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d74cf-ae5b-4fe2-8a39-dee258a6504b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:29.633511 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d74cf-ae5b-4fe2-8a39-dee258a6504b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:29.633789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d74cf-ae5b-4fe2-8a39-dee258a6504b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:29.634060 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d74cf-ae5b-4fe2-8a39-dee258a6504b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:29.644416 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d74cf-ae5b-4fe2-8a39-dee258a6504b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:26:29.652174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-d22d74cf-ae5b-4fe2-8a39-dee258a6504b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:26:29.652174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:26:29.652174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:26:29.652174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:26:29.652174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:26:29.652174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:26:29.652174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:26:29.652174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:26:29.652174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:26:29.652174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:26:29.652174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:26:29.652174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:26:29.652174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:26:29.652174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:26:29.652174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:26:29.652174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:26:29.653348 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:26:29.653348 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:26:29.653348 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:26:29.653348 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:26:29.653348 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:26:29.653348 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:26:29.653348 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:26:29.653348 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:26:29.653348 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:26:29.653348 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:26:29.653348 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:26:29.653348 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:26:29.653348 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:26:29.653348 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:26:29.653348 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:26:29.653348 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:26:29.655117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:26:29.655117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:26:29.655117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:26:29.655117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:26:29.655117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:26:29.655117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:26:29.655117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:26:29.655117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:26:29.655117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:26:29.655117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:26:29.655117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:26:29.655117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:26:29.655117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 580/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:26:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:26:29.660094 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5948c-8e3e-4f29-9b68-ca8a8be043c6 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:29.713262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:29.757095 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5948c-8e3e-4f29-9b68-ca8a8be043c6 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:29.789899 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865d218-daf6-4cef-a8e7-f08e4d19ad20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:29.790551 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865d218-daf6-4cef-a8e7-f08e4d19ad20 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:29.790835 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865d218-daf6-4cef-a8e7-f08e4d19ad20 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:29.791109 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865d218-daf6-4cef-a8e7-f08e4d19ad20 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:29.791871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865d218-daf6-4cef-a8e7-f08e4d19ad20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:26:29.792177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865d218-daf6-4cef-a8e7-f08e4d19ad20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:29.793530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865d218-daf6-4cef-a8e7-f08e4d19ad20 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:29.801858 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 577/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:26:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:29.807519 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d74cf-ae5b-4fe2-8a39-dee258a6504b None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:29.903276 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d74cf-ae5b-4fe2-8a39-dee258a6504b None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:29.935415 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922995c9-0423-4ae2-996c-3130a0cc1b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:29.936386 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922995c9-0423-4ae2-996c-3130a0cc1b03 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:29.936665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922995c9-0423-4ae2-996c-3130a0cc1b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:29.936932 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922995c9-0423-4ae2-996c-3130a0cc1b03 None admin] PATH_INFO: `/v3/users/d118e5dd9188403ca1aeb553cbcaa9c5` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:29.938081 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922995c9-0423-4ae2-996c-3130a0cc1b03 None admin] RBAC: Authorizing `identity:get_user(user_id=d118e5dd9188403ca1aeb553cbcaa9c5)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:29.939489 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922995c9-0423-4ae2-996c-3130a0cc1b03 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:29.941096 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 581/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:26:29 2022] GET /identity/v3/users/d118e5dd9188403ca1aeb553cbcaa9c5 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:29.946670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865d218-daf6-4cef-a8e7-f08e4d19ad20 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:30.043731 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865d218-daf6-4cef-a8e7-f08e4d19ad20 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:30.059171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:30.077239 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35d0f7-10cf-4947-b418-f1d2df3486a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:30.077868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35d0f7-10cf-4947-b418-f1d2df3486a4 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:30.078153 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35d0f7-10cf-4947-b418-f1d2df3486a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:30.078426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35d0f7-10cf-4947-b418-f1d2df3486a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:30.143070 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35d0f7-10cf-4947-b418-f1d2df3486a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:30.144527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35d0f7-10cf-4947-b418-f1d2df3486a4 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:30.241457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 578/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:26:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 10:26:30.247246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922995c9-0423-4ae2-996c-3130a0cc1b03 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:30.286392 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:30.343405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922995c9-0423-4ae2-996c-3130a0cc1b03 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:30.375558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01b11a-decb-437f-9584-0a02b40d206b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:30.376417 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01b11a-decb-437f-9584-0a02b40d206b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:30.376695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01b11a-decb-437f-9584-0a02b40d206b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:30.376962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01b11a-decb-437f-9584-0a02b40d206b None admin] PATH_INFO: `/v3/projects/972cb8083d0c4f3397d89628d338a1d9` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:30.378129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c01b11a-decb-437f-9584-0a02b40d206b None admin] RBAC: Authorizing `identity:get_project(project_id=972cb8083d0c4f3397d89628d338a1d9)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:30.379546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c01b11a-decb-437f-9584-0a02b40d206b None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:30.381939 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 582/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:26:30 2022] GET /identity/v3/projects/972cb8083d0c4f3397d89628d338a1d9 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:30.387288 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35d0f7-10cf-4947-b418-f1d2df3486a4 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:30.405073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:30.485219 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35d0f7-10cf-4947-b418-f1d2df3486a4 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:30.517635 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b3a3f-1474-4388-817a-6311378ffa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:30.518222 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b3a3f-1474-4388-817a-6311378ffa4f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:30.518504 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b3a3f-1474-4388-817a-6311378ffa4f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:30.518778 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b3a3f-1474-4388-817a-6311378ffa4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:30.519603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b3a3f-1474-4388-817a-6311378ffa4f None admin] RBAC: Adding query filter params (role.id=0f929204eb7b4ea8859338ed2d3ff75b, scope.project.id=972cb8083d0c4f3397d89628d338a1d9, user.id=d118e5dd9188403ca1aeb553cbcaa9c5) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:26:30.519943 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b3a3f-1474-4388-817a-6311378ffa4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:30.521369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b3a3f-1474-4388-817a-6311378ffa4f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:30.527282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 579/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:26:30 2022] GET /identity/v3/role_assignments?user.id=d118e5dd9188403ca1aeb553cbcaa9c5&scope.project.id=972cb8083d0c4f3397d89628d338a1d9&role.id=0f929204eb7b4ea8859338ed2d3ff75b => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:32.793875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856542a-d749-458a-a279-8f9d1455bfa5 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:32.794427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856542a-d749-458a-a279-8f9d1455bfa5 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:32.794427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856542a-d749-458a-a279-8f9d1455bfa5 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:32.796871 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 583/1162] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:26:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:26:32.802161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6ec48-5f51-4f1c-8733-ed519945a55c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:32.802577 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6ec48-5f51-4f1c-8733-ed519945a55c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:32.802718 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6ec48-5f51-4f1c-8733-ed519945a55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:32.819875 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-4ee6ec48-5f51-4f1c-8733-ed519945a55c None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:26:32.821047 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4ee6ec48-5f51-4f1c-8733-ed519945a55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:26:32.885093 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 580/1163] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:26:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:26:32.908724 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856542a-d749-458a-a279-8f9d1455bfa5 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:33.010849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856542a-d749-458a-a279-8f9d1455bfa5 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:33.018635 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:33.046095 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0aca3-7781-4e94-932d-febbdeec3b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:33.046662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0aca3-7781-4e94-932d-febbdeec3b43 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:33.046943 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0aca3-7781-4e94-932d-febbdeec3b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:33.047219 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0aca3-7781-4e94-932d-febbdeec3b43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:33.111246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0aca3-7781-4e94-932d-febbdeec3b43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:33.112695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0aca3-7781-4e94-932d-febbdeec3b43 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:33.206569 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 584/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:26:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 10:26:33.212581 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6ec48-5f51-4f1c-8733-ed519945a55c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:33.224404 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:33.325211 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6ec48-5f51-4f1c-8733-ed519945a55c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:33.358904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f6994-e45a-4419-9f0a-437494a09566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:33.359694 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f6994-e45a-4419-9f0a-437494a09566 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:33.360000 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f6994-e45a-4419-9f0a-437494a09566 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:33.360273 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f6994-e45a-4419-9f0a-437494a09566 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:33.366674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f6994-e45a-4419-9f0a-437494a09566 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:33.368141 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f6994-e45a-4419-9f0a-437494a09566 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:33.382044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-490f6994-e45a-4419-9f0a-437494a09566 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 06 10:26:33.382044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:26:33.382044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 06 10:26:33.382044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 06 10:26:33.382044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:26:33.382044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:26:33.382044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:26:33.382044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:26:33.382044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 06 10:26:33.382044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:26:33.382044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:26:33.382044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:26:33.382044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:26:33.382044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:26:33.382044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:26:33.382044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:26:33.382044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:26:33.383302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:26:33.383302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:26:33.383302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:26:33.383302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:26:33.383302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:26:33.383302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:26:33.383302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:26:33.383302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:26:33.383302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:26:33.383302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:26:33.383302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:26:33.383302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:26:33.383302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:26:33.383302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:26:33.383302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:26:33.383302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:26:33.385113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:26:33.385113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:26:33.385113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:26:33.385113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:26:33.385113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:26:33.385113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:26:33.385113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:26:33.385113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:26:33.385113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:26:33.385113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:26:33.385113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 06 10:26:33.385113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:26:33.385113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 06 10:26:33.385113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:26:33.388039 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 581/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jun 6 10:26:33 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:26:33.389535 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0aca3-7781-4e94-932d-febbdeec3b43 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:33.397747 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:33.485941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0aca3-7781-4e94-932d-febbdeec3b43 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:33.518514 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ceb710-5290-47bc-bddd-816b526f36d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:33.519150 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ceb710-5290-47bc-bddd-816b526f36d8 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:33.519434 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ceb710-5290-47bc-bddd-816b526f36d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:33.519703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ceb710-5290-47bc-bddd-816b526f36d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:33.520427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ceb710-5290-47bc-bddd-816b526f36d8 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:26:33.520728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ceb710-5290-47bc-bddd-816b526f36d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:33.522323 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ceb710-5290-47bc-bddd-816b526f36d8 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:33.529146 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 585/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Jun 6 10:26:33 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:35.774879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8915cc-d419-4a42-a522-5213c59a1d8b None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:35.775439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8915cc-d419-4a42-a522-5213c59a1d8b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:35.775439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8915cc-d419-4a42-a522-5213c59a1d8b None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:35.776808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 582/1167] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:26:35.782432 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828ffb8-e228-400d-9ebc-1cbdd9989503 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:35.782817 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828ffb8-e228-400d-9ebc-1cbdd9989503 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:35.782981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828ffb8-e228-400d-9ebc-1cbdd9989503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:35.799858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-9828ffb8-e228-400d-9ebc-1cbdd9989503 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:26:35.801026 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-9828ffb8-e228-400d-9ebc-1cbdd9989503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:26:35.861171 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:35.866120 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 586/1168] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:26:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 10:26:35.889441 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8915cc-d419-4a42-a522-5213c59a1d8b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:35.958086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:35.988940 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8915cc-d419-4a42-a522-5213c59a1d8b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:36.020889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b34943-adcc-44da-b8c4-54a1f170fc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:36.021532 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b34943-adcc-44da-b8c4-54a1f170fc90 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:36.022079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b34943-adcc-44da-b8c4-54a1f170fc90 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:36.022354 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b34943-adcc-44da-b8c4-54a1f170fc90 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:36.023073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b34943-adcc-44da-b8c4-54a1f170fc90 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:36.024463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b34943-adcc-44da-b8c4-54a1f170fc90 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:36.038955 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 583/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Mon Jun 6 10:26:35 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:26:38.300315 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75679690-c96d-485e-9474-b8a9918e0f6b None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:38.300868 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75679690-c96d-485e-9474-b8a9918e0f6b None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:38.300868 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75679690-c96d-485e-9474-b8a9918e0f6b None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:38.302226 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 587/1170] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:26:38.307422 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc53f7-60cf-471e-bed2-66ce76e2acf3 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:38.307881 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc53f7-60cf-471e-bed2-66ce76e2acf3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:38.308007 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc53f7-60cf-471e-bed2-66ce76e2acf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:38.324461 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-44cc53f7-60cf-471e-bed2-66ce76e2acf3 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:26:38.325634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-44cc53f7-60cf-471e-bed2-66ce76e2acf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:26:38.393359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 584/1171] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:26:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:26:38.417562 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75679690-c96d-485e-9474-b8a9918e0f6b None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:38.517527 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75679690-c96d-485e-9474-b8a9918e0f6b None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:38.550073 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c394302-294c-4583-bd4d-ada9a84b5ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:38.550958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c394302-294c-4583-bd4d-ada9a84b5ece None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:38.551187 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c394302-294c-4583-bd4d-ada9a84b5ece None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:38.551451 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c394302-294c-4583-bd4d-ada9a84b5ece None admin] PATH_INFO: `/v3/domains/295f2d89edfc4f798105223a513a4404` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:38.562203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c394302-294c-4583-bd4d-ada9a84b5ece None admin] RBAC: Authorizing `identity:get_domain(domain_id=295f2d89edfc4f798105223a513a4404)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:38.563618 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c394302-294c-4583-bd4d-ada9a84b5ece None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:38.565244 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 588/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Mon Jun 6 10:26:38 2022] GET /identity/v3/domains/295f2d89edfc4f798105223a513a4404 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:38.571081 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc53f7-60cf-471e-bed2-66ce76e2acf3 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:38.626721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:38.665844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc53f7-60cf-471e-bed2-66ce76e2acf3 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:38.698318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696dc4fb-809a-4da4-9770-773bdce347ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:38.698959 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696dc4fb-809a-4da4-9770-773bdce347ee None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:38.699238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696dc4fb-809a-4da4-9770-773bdce347ee None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:38.699508 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696dc4fb-809a-4da4-9770-773bdce347ee None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:38.700237 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696dc4fb-809a-4da4-9770-773bdce347ee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:38.702478 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696dc4fb-809a-4da4-9770-773bdce347ee None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:38.725925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 585/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jun 6 10:26:38 2022] POST /identity/v3/projects => generated 361 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:26:40.999732 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fc56c-2dcd-45af-92c8-e90d0f5508e1 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:41.000287 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fc56c-2dcd-45af-92c8-e90d0f5508e1 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:41.000287 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fc56c-2dcd-45af-92c8-e90d0f5508e1 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:41.001653 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 589/1174] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:26:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:26:41.007002 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb0458-d3ce-4f66-9316-b0db28327278 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:41.007417 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb0458-d3ce-4f66-9316-b0db28327278 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:41.007561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb0458-d3ce-4f66-9316-b0db28327278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:41.025055 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-11bb0458-d3ce-4f66-9316-b0db28327278 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:26:41.027111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-11bb0458-d3ce-4f66-9316-b0db28327278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:26:41.091234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 586/1175] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:26:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:26:41.116189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fc56c-2dcd-45af-92c8-e90d0f5508e1 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:41.174519 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:41.215129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fc56c-2dcd-45af-92c8-e90d0f5508e1 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:41.247547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f987c-6ca4-4d0d-9fd7-3fb2a53a7983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:41.248838 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f987c-6ca4-4d0d-9fd7-3fb2a53a7983 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:41.249123 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f987c-6ca4-4d0d-9fd7-3fb2a53a7983 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:41.249395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f987c-6ca4-4d0d-9fd7-3fb2a53a7983 None admin] PATH_INFO: `/v3/domains/295f2d89edfc4f798105223a513a4404` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:41.250603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f987c-6ca4-4d0d-9fd7-3fb2a53a7983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=295f2d89edfc4f798105223a513a4404)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:41.251982 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f987c-6ca4-4d0d-9fd7-3fb2a53a7983 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:41.253568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 590/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Mon Jun 6 10:26:41 2022] GET /identity/v3/domains/295f2d89edfc4f798105223a513a4404 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:41.259190 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb0458-d3ce-4f66-9316-b0db28327278 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:41.301807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:41.356624 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb0458-d3ce-4f66-9316-b0db28327278 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:41.390153 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa79b6-367e-4e88-b172-bf11f78d2b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:41.390815 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa79b6-367e-4e88-b172-bf11f78d2b02 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:41.391092 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa79b6-367e-4e88-b172-bf11f78d2b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:41.391358 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa79b6-367e-4e88-b172-bf11f78d2b02 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:41.392123 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa79b6-367e-4e88-b172-bf11f78d2b02 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:41.393525 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa79b6-367e-4e88-b172-bf11f78d2b02 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:41.413036 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 587/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jun 6 10:26:41 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:26:43.706979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19627faa-8786-4289-95ac-b00c6d089a9c None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:43.707514 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19627faa-8786-4289-95ac-b00c6d089a9c None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:43.707514 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19627faa-8786-4289-95ac-b00c6d089a9c None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:43.708907 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 591/1178] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:26:43.714032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58acae2-165d-416a-b0fd-85096c4ed650 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:43.714467 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58acae2-165d-416a-b0fd-85096c4ed650 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:43.714589 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58acae2-165d-416a-b0fd-85096c4ed650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:43.731841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-f58acae2-165d-416a-b0fd-85096c4ed650 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:26:43.733092 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f58acae2-165d-416a-b0fd-85096c4ed650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:26:43.796936 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 588/1179] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:26:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:26:43.821849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19627faa-8786-4289-95ac-b00c6d089a9c None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:43.921500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19627faa-8786-4289-95ac-b00c6d089a9c None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:43.946402 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:43.954604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185058fb-ca78-4799-8ea2-7895df22809e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:43.955366 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185058fb-ca78-4799-8ea2-7895df22809e None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:43.955643 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185058fb-ca78-4799-8ea2-7895df22809e None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:43.955941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185058fb-ca78-4799-8ea2-7895df22809e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:43.966041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185058fb-ca78-4799-8ea2-7895df22809e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:26:43.974229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-185058fb-ca78-4799-8ea2-7895df22809e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:26:43.974229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:26:43.974229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:26:43.974229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:26:43.974229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:26:43.974229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:26:43.974229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:26:43.974229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:26:43.974229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:26:43.974229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:26:43.974229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:26:43.974229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:26:43.974229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:26:43.974229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:26:43.974229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:26:43.974229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:26:43.975316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:26:43.975316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:26:43.975316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:26:43.975316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:26:43.975316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:26:43.975316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:26:43.975316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:26:43.975316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:26:43.975316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:26:43.975316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:26:43.975316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:26:43.975316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:26:43.975316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:26:43.975316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:26:43.975316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:26:43.975316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:26:43.976327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:26:43.976327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:26:43.976327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:26:43.976327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:26:43.976327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:26:43.976327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:26:43.976327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:26:43.976327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:26:43.976327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:26:43.976327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:26:43.976327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:26:43.976327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:26:43.976327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 592/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:26:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:26:43.981462 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58acae2-165d-416a-b0fd-85096c4ed650 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:44.008302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:44.077228 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58acae2-165d-416a-b0fd-85096c4ed650 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:44.109462 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906eb6e-fcfb-476a-94fb-d5cd11614384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:44.110111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906eb6e-fcfb-476a-94fb-d5cd11614384 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:44.110392 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906eb6e-fcfb-476a-94fb-d5cd11614384 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:44.110665 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906eb6e-fcfb-476a-94fb-d5cd11614384 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:44.111401 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906eb6e-fcfb-476a-94fb-d5cd11614384 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:26:44.111707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906eb6e-fcfb-476a-94fb-d5cd11614384 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:44.113091 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906eb6e-fcfb-476a-94fb-d5cd11614384 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:44.120801 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 589/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:26:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:44.126581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185058fb-ca78-4799-8ea2-7895df22809e None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:44.224479 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185058fb-ca78-4799-8ea2-7895df22809e None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:44.256746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78683a6d-3fd3-4010-970b-103ce9d05686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:44.257594 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78683a6d-3fd3-4010-970b-103ce9d05686 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:44.257872 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78683a6d-3fd3-4010-970b-103ce9d05686 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:44.258142 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78683a6d-3fd3-4010-970b-103ce9d05686 None admin] PATH_INFO: `/v3/users/5cdaf036c04a4192a3694105f1c681cc` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:44.272628 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78683a6d-3fd3-4010-970b-103ce9d05686 None admin] RBAC: Authorizing `identity:get_user(user_id=5cdaf036c04a4192a3694105f1c681cc)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:44.274039 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78683a6d-3fd3-4010-970b-103ce9d05686 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:44.275633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 593/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:26:44 2022] GET /identity/v3/users/5cdaf036c04a4192a3694105f1c681cc => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:44.281914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906eb6e-fcfb-476a-94fb-d5cd11614384 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:44.368490 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:44.378528 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906eb6e-fcfb-476a-94fb-d5cd11614384 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:44.412938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444aa206-2e1b-4ab3-9cf0-5c548871f8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:44.413577 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444aa206-2e1b-4ab3-9cf0-5c548871f8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:44.413861 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444aa206-2e1b-4ab3-9cf0-5c548871f8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:44.414136 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444aa206-2e1b-4ab3-9cf0-5c548871f8f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:44.478643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444aa206-2e1b-4ab3-9cf0-5c548871f8f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:44.480031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444aa206-2e1b-4ab3-9cf0-5c548871f8f0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:44.563973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:44.574059 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 590/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:26:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 10:26:44.580515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78683a6d-3fd3-4010-970b-103ce9d05686 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:44.628778 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:44.681814 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78683a6d-3fd3-4010-970b-103ce9d05686 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:44.715400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bdd0ce-146b-4e77-b2fb-4470b8bd8304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:44.716284 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdd0ce-146b-4e77-b2fb-4470b8bd8304 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:44.716565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdd0ce-146b-4e77-b2fb-4470b8bd8304 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:44.716830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdd0ce-146b-4e77-b2fb-4470b8bd8304 None admin] PATH_INFO: `/v3/projects/355992969019433db3490fb2caedd398` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:44.718757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bdd0ce-146b-4e77-b2fb-4470b8bd8304 None admin] RBAC: Authorizing `identity:get_project(project_id=355992969019433db3490fb2caedd398)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:44.721228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bdd0ce-146b-4e77-b2fb-4470b8bd8304 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:44.723239 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 594/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:26:44 2022] GET /identity/v3/projects/355992969019433db3490fb2caedd398 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:44.729416 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444aa206-2e1b-4ab3-9cf0-5c548871f8f0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:44.824000 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444aa206-2e1b-4ab3-9cf0-5c548871f8f0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:44.858385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b8ef8-d011-4038-8d52-c095103c88cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:44.859018 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b8ef8-d011-4038-8d52-c095103c88cf None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:44.859299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b8ef8-d011-4038-8d52-c095103c88cf None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:44.859571 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b8ef8-d011-4038-8d52-c095103c88cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:44.860422 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b8ef8-d011-4038-8d52-c095103c88cf None admin] RBAC: Adding query filter params (role.id=0f929204eb7b4ea8859338ed2d3ff75b, scope.project.id=355992969019433db3490fb2caedd398, user.id=5cdaf036c04a4192a3694105f1c681cc) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:26:44.860734 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b8ef8-d011-4038-8d52-c095103c88cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:44.862151 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b8ef8-d011-4038-8d52-c095103c88cf None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:44.867418 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 591/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:26:44 2022] GET /identity/v3/role_assignments?user.id=5cdaf036c04a4192a3694105f1c681cc&scope.project.id=355992969019433db3490fb2caedd398&role.id=0f929204eb7b4ea8859338ed2d3ff75b => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:47.059449 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea201a-3117-4635-849d-c6a74bdb24cf None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:47.060587 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea201a-3117-4635-849d-c6a74bdb24cf None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:47.060587 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea201a-3117-4635-849d-c6a74bdb24cf None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:47.061983 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 595/1186] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:26:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:26:47.067228 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0d97c-12bf-4ad5-8757-2cf8e6ba869b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:47.067602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0d97c-12bf-4ad5-8757-2cf8e6ba869b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:47.067794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0d97c-12bf-4ad5-8757-2cf8e6ba869b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:47.084543 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-73b0d97c-12bf-4ad5-8757-2cf8e6ba869b None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:26:47.085710 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-73b0d97c-12bf-4ad5-8757-2cf8e6ba869b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:26:47.132901 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:47.150273 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 592/1187] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:26:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:26:47.173809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea201a-3117-4635-849d-c6a74bdb24cf None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:47.273361 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea201a-3117-4635-849d-c6a74bdb24cf None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:47.287303 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:47.306562 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92badeb1-cc57-440e-aee0-9eee2fc39d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:47.307330 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92badeb1-cc57-440e-aee0-9eee2fc39d24 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:47.307611 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92badeb1-cc57-440e-aee0-9eee2fc39d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:47.307915 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92badeb1-cc57-440e-aee0-9eee2fc39d24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:47.317708 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92badeb1-cc57-440e-aee0-9eee2fc39d24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:26:47.325196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-92badeb1-cc57-440e-aee0-9eee2fc39d24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:26:47.325196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:26:47.325196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:26:47.325196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:26:47.325196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:26:47.325196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:26:47.325196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:26:47.325196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:26:47.325196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:26:47.325196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:26:47.325196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:26:47.325196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:26:47.325196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:26:47.325196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:26:47.325196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:26:47.325196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:26:47.326886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:26:47.326886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:26:47.326886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:26:47.326886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:26:47.326886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:26:47.326886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:26:47.326886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:26:47.326886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:26:47.326886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:26:47.326886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:26:47.326886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:26:47.326886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:26:47.326886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:26:47.326886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:26:47.326886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:26:47.326886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:26:47.327973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:26:47.327973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:26:47.327973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:26:47.327973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:26:47.327973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:26:47.327973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:26:47.327973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:26:47.327973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:26:47.327973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:26:47.327973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:26:47.327973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:26:47.327973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:26:47.327973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 596/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:26:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:26:47.333150 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b0d97c-12bf-4ad5-8757-2cf8e6ba869b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:47.431987 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b0d97c-12bf-4ad5-8757-2cf8e6ba869b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:47.464704 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a06f2-c913-48a5-a6b1-f2fb62d3ba0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:47.465359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a06f2-c913-48a5-a6b1-f2fb62d3ba0f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:47.465640 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a06f2-c913-48a5-a6b1-f2fb62d3ba0f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:47.466519 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a06f2-c913-48a5-a6b1-f2fb62d3ba0f None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:47.467264 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a06f2-c913-48a5-a6b1-f2fb62d3ba0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:26:47.467567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a06f2-c913-48a5-a6b1-f2fb62d3ba0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:47.468956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a06f2-c913-48a5-a6b1-f2fb62d3ba0f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:47.476781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 593/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:26:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:47.482774 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92badeb1-cc57-440e-aee0-9eee2fc39d24 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:47.579889 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92badeb1-cc57-440e-aee0-9eee2fc39d24 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:47.611848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9de506-375b-490b-95a9-d2b83d7632a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:47.612702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9de506-375b-490b-95a9-d2b83d7632a8 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:47.612986 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9de506-375b-490b-95a9-d2b83d7632a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:47.613259 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9de506-375b-490b-95a9-d2b83d7632a8 None admin] PATH_INFO: `/v3/users/5cdaf036c04a4192a3694105f1c681cc` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:47.614419 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9de506-375b-490b-95a9-d2b83d7632a8 None admin] RBAC: Authorizing `identity:get_user(user_id=5cdaf036c04a4192a3694105f1c681cc)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:47.615858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9de506-375b-490b-95a9-d2b83d7632a8 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:47.617472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 597/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:26:47 2022] GET /identity/v3/users/5cdaf036c04a4192a3694105f1c681cc => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:47.624229 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a06f2-c913-48a5-a6b1-f2fb62d3ba0f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:47.669765 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:47.728815 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a06f2-c913-48a5-a6b1-f2fb62d3ba0f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:47.761064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1a3f9-cec2-4327-9df4-38d299efdc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:47.761942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1a3f9-cec2-4327-9df4-38d299efdc10 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:47.762219 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1a3f9-cec2-4327-9df4-38d299efdc10 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:47.762488 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1a3f9-cec2-4327-9df4-38d299efdc10 None admin] PATH_INFO: `/v3/projects/355992969019433db3490fb2caedd398` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:47.764513 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e1a3f9-cec2-4327-9df4-38d299efdc10 None admin] RBAC: Authorizing `identity:get_project(project_id=355992969019433db3490fb2caedd398)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:47.765936 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e1a3f9-cec2-4327-9df4-38d299efdc10 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:47.767613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 594/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:26:47 2022] GET /identity/v3/projects/355992969019433db3490fb2caedd398 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:47.773789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9de506-375b-490b-95a9-d2b83d7632a8 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:47.816109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:47.881184 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9de506-375b-490b-95a9-d2b83d7632a8 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:47.913092 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157e10a-700b-4f81-8f5c-ca8f712641ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:47.913958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157e10a-700b-4f81-8f5c-ca8f712641ba None admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:47.914237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157e10a-700b-4f81-8f5c-ca8f712641ba None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:47.914508 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157e10a-700b-4f81-8f5c-ca8f712641ba None admin] PATH_INFO: `/v3/projects/355992969019433db3490fb2caedd398/users/5cdaf036c04a4192a3694105f1c681cc/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:47.916439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9157e10a-700b-4f81-8f5c-ca8f712641ba None admin] RBAC: Authorizing `identity:create_grant(project_id=355992969019433db3490fb2caedd398, user_id=5cdaf036c04a4192a3694105f1c681cc, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:47.917853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9157e10a-700b-4f81-8f5c-ca8f712641ba None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:47.930731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 598/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:26:47 2022] PUT /identity/v3/projects/355992969019433db3490fb2caedd398/users/5cdaf036c04a4192a3694105f1c681cc/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:26:50.174687 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d701836-eb15-4bcd-8796-03d47ff0004b None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:50.175219 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d701836-eb15-4bcd-8796-03d47ff0004b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:50.175219 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d701836-eb15-4bcd-8796-03d47ff0004b None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:50.176605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 595/1193] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:26:50.181850 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76905020-4fdf-4899-b40d-ef5db3869545 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:50.182231 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76905020-4fdf-4899-b40d-ef5db3869545 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:50.182409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76905020-4fdf-4899-b40d-ef5db3869545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:50.199838 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-76905020-4fdf-4899-b40d-ef5db3869545 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:26:50.201118 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-76905020-4fdf-4899-b40d-ef5db3869545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:26:50.265858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 599/1194] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:26:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:26:50.290404 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d701836-eb15-4bcd-8796-03d47ff0004b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:50.389759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d701836-eb15-4bcd-8796-03d47ff0004b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:50.403101 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:50.423205 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b753f9-cea5-4fb3-b20c-1f082646a797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:50.424021 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b753f9-cea5-4fb3-b20c-1f082646a797 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:50.424307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b753f9-cea5-4fb3-b20c-1f082646a797 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:50.424579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b753f9-cea5-4fb3-b20c-1f082646a797 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:50.435572 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b753f9-cea5-4fb3-b20c-1f082646a797 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:26:50.443253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-f1b753f9-cea5-4fb3-b20c-1f082646a797 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:26:50.443253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:26:50.443253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:26:50.443253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:26:50.443253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:26:50.443253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:26:50.443253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:26:50.443253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:26:50.443253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:26:50.443253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:26:50.443253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:26:50.443253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:26:50.443253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:26:50.443253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:26:50.443253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:26:50.443253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:26:50.444345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:26:50.444345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:26:50.444345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:26:50.444345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:26:50.444345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:26:50.444345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:26:50.444345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:26:50.444345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:26:50.444345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:26:50.444345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:26:50.444345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:26:50.444345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:26:50.444345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:26:50.444345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:26:50.444345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:26:50.444345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:26:50.445399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:26:50.445399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:26:50.445399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:26:50.445399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:26:50.445399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:26:50.445399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:26:50.445399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:26:50.445399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:26:50.445399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:26:50.445399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:26:50.445399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:26:50.445399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:26:50.445399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 596/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:26:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:26:50.450582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76905020-4fdf-4899-b40d-ef5db3869545 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:50.480733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:50.548457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76905020-4fdf-4899-b40d-ef5db3869545 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:50.582019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ddfce-061b-461f-8eea-f025ba14c73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:50.582670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ddfce-061b-461f-8eea-f025ba14c73d None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:50.582949 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ddfce-061b-461f-8eea-f025ba14c73d None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:50.583216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ddfce-061b-461f-8eea-f025ba14c73d None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:50.583967 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ddfce-061b-461f-8eea-f025ba14c73d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:26:50.584266 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ddfce-061b-461f-8eea-f025ba14c73d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:50.585620 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ddfce-061b-461f-8eea-f025ba14c73d None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:50.593483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 600/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:26:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:50.599163 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b753f9-cea5-4fb3-b20c-1f082646a797 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:50.694018 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b753f9-cea5-4fb3-b20c-1f082646a797 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:50.726655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9484d2-c8d4-46c9-b75e-bd9f0a77ce7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:50.727519 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9484d2-c8d4-46c9-b75e-bd9f0a77ce7f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:50.727823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9484d2-c8d4-46c9-b75e-bd9f0a77ce7f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:50.728106 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9484d2-c8d4-46c9-b75e-bd9f0a77ce7f None admin] PATH_INFO: `/v3/users/5cdaf036c04a4192a3694105f1c681cc` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:50.729288 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9484d2-c8d4-46c9-b75e-bd9f0a77ce7f None admin] RBAC: Authorizing `identity:get_user(user_id=5cdaf036c04a4192a3694105f1c681cc)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:50.730722 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9484d2-c8d4-46c9-b75e-bd9f0a77ce7f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:50.733496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 597/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jun 6 10:26:50 2022] GET /identity/v3/users/5cdaf036c04a4192a3694105f1c681cc => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:50.738262 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ddfce-061b-461f-8eea-f025ba14c73d None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:50.829776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:50.836923 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ddfce-061b-461f-8eea-f025ba14c73d None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:50.869213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237afb74-9662-4cf7-9072-56c5429fd2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:50.869843 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237afb74-9662-4cf7-9072-56c5429fd2cc None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:50.870125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237afb74-9662-4cf7-9072-56c5429fd2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:50.870399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237afb74-9662-4cf7-9072-56c5429fd2cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:50.936179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237afb74-9662-4cf7-9072-56c5429fd2cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:50.937616 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237afb74-9662-4cf7-9072-56c5429fd2cc None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:51.025529 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:51.033127 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 601/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:26:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 10:26:51.038968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9484d2-c8d4-46c9-b75e-bd9f0a77ce7f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:51.083678 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:26:51.138715 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9484d2-c8d4-46c9-b75e-bd9f0a77ce7f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:51.171667 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96142f-c307-4f26-bc2f-963b6d76bb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:51.172551 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96142f-c307-4f26-bc2f-963b6d76bb67 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:51.172833 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96142f-c307-4f26-bc2f-963b6d76bb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:51.173104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96142f-c307-4f26-bc2f-963b6d76bb67 None admin] PATH_INFO: `/v3/projects/355992969019433db3490fb2caedd398` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:51.174267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96142f-c307-4f26-bc2f-963b6d76bb67 None admin] RBAC: Authorizing `identity:get_project(project_id=355992969019433db3490fb2caedd398)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:51.175700 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96142f-c307-4f26-bc2f-963b6d76bb67 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:51.177405 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 598/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jun 6 10:26:51 2022] GET /identity/v3/projects/355992969019433db3490fb2caedd398 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:26:51.183486 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237afb74-9662-4cf7-9072-56c5429fd2cc None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:26:51.279671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237afb74-9662-4cf7-9072-56c5429fd2cc None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:26:51.312228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06312700-97a3-43fa-9440-b1c81d2844db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:26:51.312855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06312700-97a3-43fa-9440-b1c81d2844db None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:26:51.313132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06312700-97a3-43fa-9440-b1c81d2844db None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:26:51.313399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06312700-97a3-43fa-9440-b1c81d2844db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:26:51.314203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06312700-97a3-43fa-9440-b1c81d2844db None admin] RBAC: Adding query filter params (role.id=0f929204eb7b4ea8859338ed2d3ff75b, scope.project.id=355992969019433db3490fb2caedd398, user.id=5cdaf036c04a4192a3694105f1c681cc) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:26:51.314504 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06312700-97a3-43fa-9440-b1c81d2844db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:26:51.315927 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06312700-97a3-43fa-9440-b1c81d2844db None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:26:51.321488 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 602/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:26:51 2022] GET /identity/v3/role_assignments?user.id=5cdaf036c04a4192a3694105f1c681cc&scope.project.id=355992969019433db3490fb2caedd398&role.id=0f929204eb7b4ea8859338ed2d3ff75b => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:27:01.166997 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=45 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:27:01.318953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=71 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:27:52.438814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d624e3-e40e-4465-8128-fb6ca89a9cc1 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:27:52.438814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d624e3-e40e-4465-8128-fb6ca89a9cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:27:52.438814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d624e3-e40e-4465-8128-fb6ca89a9cc1 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:27:52.439890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 599/1201] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:27:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:27:52.445485 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48502c20-02ac-4feb-a19e-f082493123e1 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:27:52.446011 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48502c20-02ac-4feb-a19e-f082493123e1 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:27:52.446123 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48502c20-02ac-4feb-a19e-f082493123e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:27:52.464355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-48502c20-02ac-4feb-a19e-f082493123e1 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:27:52.465687 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-48502c20-02ac-4feb-a19e-f082493123e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:27:52.559808 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 603/1202] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:27:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 10:27:52.579735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d624e3-e40e-4465-8128-fb6ca89a9cc1 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:27:52.709530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d624e3-e40e-4465-8128-fb6ca89a9cc1 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:27:52.751454 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f140cb6-4dca-45bd-98a5-f1bd249fadc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:27:52.752539 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f140cb6-4dca-45bd-98a5-f1bd249fadc1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:27:52.752842 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f140cb6-4dca-45bd-98a5-f1bd249fadc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:27:52.753186 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f140cb6-4dca-45bd-98a5-f1bd249fadc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:27:52.765392 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f140cb6-4dca-45bd-98a5-f1bd249fadc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:27:52.766887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f140cb6-4dca-45bd-98a5-f1bd249fadc1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:27:52.793161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-8f140cb6-4dca-45bd-98a5-f1bd249fadc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:27:52.793161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:27:52.793161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:27:52.793161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:27:52.793161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:27:52.793161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:27:52.793161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:27:52.793161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:27:52.793161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:27:52.793161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:27:52.793161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:27:52.793161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:27:52.793161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:27:52.793161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:27:52.793161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:27:52.794409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:27:52.794409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:27:52.794409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:27:52.794409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:27:52.794409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:27:52.794409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:27:52.794409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:27:52.794409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:27:52.794409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:27:52.794409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:27:52.794409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:27:52.794409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:27:52.794409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:27:52.794409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:27:52.794409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:27:52.794409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:27:52.794409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:27:52.795900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:27:52.795900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:27:52.795900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:27:52.795900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:27:52.795900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 600/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:27:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:27:52.801012 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48502c20-02ac-4feb-a19e-f082493123e1 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:27:52.930339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48502c20-02ac-4feb-a19e-f082493123e1 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:27:52.966561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:27:52.977641 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8dd8bb-dba3-498f-bb6d-097ffc97ec46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:27:52.978432 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8dd8bb-dba3-498f-bb6d-097ffc97ec46 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:27:52.978735 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8dd8bb-dba3-498f-bb6d-097ffc97ec46 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:27:52.979012 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8dd8bb-dba3-498f-bb6d-097ffc97ec46 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:27:52.979759 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8dd8bb-dba3-498f-bb6d-097ffc97ec46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:27:52.980112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8dd8bb-dba3-498f-bb6d-097ffc97ec46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:27:52.981566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8dd8bb-dba3-498f-bb6d-097ffc97ec46 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:27:53.000923 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 604/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:27:53.001876 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f140cb6-4dca-45bd-98a5-f1bd249fadc1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:27:53.070356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:27:53.139619 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f140cb6-4dca-45bd-98a5-f1bd249fadc1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:27:53.180385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d9d00-567e-4614-a9f8-6524c5b14359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:27:53.181919 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d9d00-567e-4614-a9f8-6524c5b14359 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:27:53.182360 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d9d00-567e-4614-a9f8-6524c5b14359 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:27:53.182642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d9d00-567e-4614-a9f8-6524c5b14359 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:27:53.183353 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d9d00-567e-4614-a9f8-6524c5b14359 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:27:53.184907 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d9d00-567e-4614-a9f8-6524c5b14359 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:27:53.218667 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 601/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jun 6 10:27:53 2022] POST /identity/v3/users => generated 251 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:27:55.685224 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf941a-9557-4cee-8f35-171974eb66e2 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:27:55.685861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf941a-9557-4cee-8f35-171974eb66e2 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:27:55.685861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf941a-9557-4cee-8f35-171974eb66e2 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:27:55.687658 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 605/1206] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:27:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:27:55.692895 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4b886-8d4c-45c7-97ce-7e50b78fdc48 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:27:55.693312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4b886-8d4c-45c7-97ce-7e50b78fdc48 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:27:55.693489 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4b886-8d4c-45c7-97ce-7e50b78fdc48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:27:55.723308 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-80d4b886-8d4c-45c7-97ce-7e50b78fdc48 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:27:55.724668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-80d4b886-8d4c-45c7-97ce-7e50b78fdc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:27:55.805615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 602/1207] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:27:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:27:55.825007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf941a-9557-4cee-8f35-171974eb66e2 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:27:55.958046 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf941a-9557-4cee-8f35-171974eb66e2 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:27:56.002155 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1889ebda-65c7-4d19-83c7-2a03ee685f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:27:56.003018 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1889ebda-65c7-4d19-83c7-2a03ee685f12 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:27:56.003305 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1889ebda-65c7-4d19-83c7-2a03ee685f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:27:56.003673 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1889ebda-65c7-4d19-83c7-2a03ee685f12 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:27:56.015567 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1889ebda-65c7-4d19-83c7-2a03ee685f12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:27:56.024019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-1889ebda-65c7-4d19-83c7-2a03ee685f12 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:27:56.024019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:27:56.024019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:27:56.024019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:27:56.024019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:27:56.024019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:27:56.024019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:27:56.024019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:27:56.024019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:27:56.024019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:27:56.024019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:27:56.024019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:27:56.024019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:27:56.024019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:27:56.024019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:27:56.024019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:27:56.026617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:27:56.026617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:27:56.026617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:27:56.026617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:27:56.026617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:27:56.026617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:27:56.026617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:27:56.026617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:27:56.026617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:27:56.026617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:27:56.026617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:27:56.026617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:27:56.026617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:27:56.026617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:27:56.026617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:27:56.026617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:27:56.027588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:27:56.027588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:27:56.027588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:27:56.027588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:27:56.027588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:27:56.027588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:27:56.027588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:27:56.027588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:27:56.027588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:27:56.027588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:27:56.027588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:27:56.027588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:27:56.027588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 606/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:27:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:27:56.033375 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4b886-8d4c-45c7-97ce-7e50b78fdc48 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:27:56.084824 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:27:56.182312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4b886-8d4c-45c7-97ce-7e50b78fdc48 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:27:56.231378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a9df2-6f2d-4bc9-b5d1-00235b4fd505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:27:56.232119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a9df2-6f2d-4bc9-b5d1-00235b4fd505 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:27:56.232489 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a9df2-6f2d-4bc9-b5d1-00235b4fd505 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:27:56.234145 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a9df2-6f2d-4bc9-b5d1-00235b4fd505 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:27:56.234145 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a9df2-6f2d-4bc9-b5d1-00235b4fd505 None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:27:56.234145 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a9df2-6f2d-4bc9-b5d1-00235b4fd505 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:27:56.235850 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a9df2-6f2d-4bc9-b5d1-00235b4fd505 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:27:56.246552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 603/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jun 6 10:27:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:27:56.252335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1889ebda-65c7-4d19-83c7-2a03ee685f12 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:27:56.314069 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:27:56.371442 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1889ebda-65c7-4d19-83c7-2a03ee685f12 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:27:56.411750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222c122-d39e-4862-a150-e3b24bf4be32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:27:56.412727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222c122-d39e-4862-a150-e3b24bf4be32 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:27:56.413091 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222c122-d39e-4862-a150-e3b24bf4be32 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:27:56.413381 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222c122-d39e-4862-a150-e3b24bf4be32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:27:56.425780 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222c122-d39e-4862-a150-e3b24bf4be32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:27:56.427285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222c122-d39e-4862-a150-e3b24bf4be32 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:27:56.453515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-e222c122-d39e-4862-a150-e3b24bf4be32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:27:56.453515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:27:56.453515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:27:56.453515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:27:56.453515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:27:56.453515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:27:56.453515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:27:56.453515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:27:56.453515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:27:56.453515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:27:56.453515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:27:56.453515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:27:56.453515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:27:56.453515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:27:56.453515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:27:56.455295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:27:56.455295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:27:56.455295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:27:56.455295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:27:56.455295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:27:56.455295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:27:56.455295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:27:56.455295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:27:56.455295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:27:56.455295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:27:56.455295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:27:56.455295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:27:56.455295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:27:56.455295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:27:56.455295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:27:56.455295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:27:56.455295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:27:56.456541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:27:56.456541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:27:56.456541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:27:56.456541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:27:56.456541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 607/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:27:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:27:56.461152 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a9df2-6f2d-4bc9-b5d1-00235b4fd505 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:27:56.592210 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:27:56.596174 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a9df2-6f2d-4bc9-b5d1-00235b4fd505 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:27:56.644272 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80977d6d-ea7d-4042-8cd1-5b0f43597ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:27:56.645294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80977d6d-ea7d-4042-8cd1-5b0f43597ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:27:56.645737 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80977d6d-ea7d-4042-8cd1-5b0f43597ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:27:56.646061 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80977d6d-ea7d-4042-8cd1-5b0f43597ac6 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:27:56.646960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80977d6d-ea7d-4042-8cd1-5b0f43597ac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:27:56.647293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80977d6d-ea7d-4042-8cd1-5b0f43597ac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:27:56.648888 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80977d6d-ea7d-4042-8cd1-5b0f43597ac6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:27:56.662894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 604/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:27:56.668702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222c122-d39e-4862-a150-e3b24bf4be32 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:27:56.781341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222c122-d39e-4862-a150-e3b24bf4be32 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:27:56.782949 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:27:56.820420 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdbc7b-dd5d-4fe9-b76a-f20d773ac69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:27:56.822105 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdbc7b-dd5d-4fe9-b76a-f20d773ac69d None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:27:56.822105 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdbc7b-dd5d-4fe9-b76a-f20d773ac69d None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:27:56.823325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdbc7b-dd5d-4fe9-b76a-f20d773ac69d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:27:56.830793 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cdbc7b-dd5d-4fe9-b76a-f20d773ac69d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:27:56.832304 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cdbc7b-dd5d-4fe9-b76a-f20d773ac69d None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:27:56.848399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-71cdbc7b-dd5d-4fe9-b76a-f20d773ac69d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 10:27:56.848399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:27:56.848399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:27:56.848399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:27:56.848399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:27:56.848399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:27:56.848399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:27:56.848399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:27:56.848399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:27:56.848399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:27:56.848399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:27:56.848399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:27:56.848399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:27:56.848399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:27:56.848399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:27:56.848399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:27:56.849574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:27:56.849574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:27:56.849574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:27:56.849574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:27:56.849574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:27:56.849574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:27:56.849574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:27:56.849574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:27:56.849574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:27:56.849574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:27:56.849574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:27:56.849574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:27:56.849574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:27:56.849574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:27:56.849574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:27:56.849574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:27:56.849574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:27:56.849574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:27:56.850785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:27:56.850785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:27:56.850785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:27:56.850785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:27:56.850785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:27:56.850785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:27:56.850785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:27:56.850785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:27:56.850785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 10:27:56.850785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:27:56.853949 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 608/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:27:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:27:56.859355 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80977d6d-ea7d-4042-8cd1-5b0f43597ac6 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:27:56.965636 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80977d6d-ea7d-4042-8cd1-5b0f43597ac6 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:27:57.001551 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d4814-60d9-4f87-98f6-1d708fe2954d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:27:57.002149 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d4814-60d9-4f87-98f6-1d708fe2954d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:27:57.002420 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d4814-60d9-4f87-98f6-1d708fe2954d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:27:57.002692 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d4814-60d9-4f87-98f6-1d708fe2954d None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:27:57.003513 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d4814-60d9-4f87-98f6-1d708fe2954d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:27:57.007081 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d4814-60d9-4f87-98f6-1d708fe2954d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:27:57.007081 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d4814-60d9-4f87-98f6-1d708fe2954d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:27:57.023299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 605/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Jun 6 10:27:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:27:57.029792 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdbc7b-dd5d-4fe9-b76a-f20d773ac69d None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:27:57.147973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdbc7b-dd5d-4fe9-b76a-f20d773ac69d None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:27:57.185748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05683a-2673-4595-97a7-e2cda40caf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:27:57.186444 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05683a-2673-4595-97a7-e2cda40caf82 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:27:57.186692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05683a-2673-4595-97a7-e2cda40caf82 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:27:57.186968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05683a-2673-4595-97a7-e2cda40caf82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:27:57.231360 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:27:57.278192 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a05683a-2673-4595-97a7-e2cda40caf82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:27:57.279680 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a05683a-2673-4595-97a7-e2cda40caf82 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:27:57.401940 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 609/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:27:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 06 10:27:57.407594 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d4814-60d9-4f87-98f6-1d708fe2954d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:27:57.429891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:27:57.536240 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d4814-60d9-4f87-98f6-1d708fe2954d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:27:57.581763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ef3ee-5e52-4440-a7d5-a0a93706d81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:27:57.582720 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ef3ee-5e52-4440-a7d5-a0a93706d81f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:27:57.583170 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ef3ee-5e52-4440-a7d5-a0a93706d81f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:27:57.583542 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ef3ee-5e52-4440-a7d5-a0a93706d81f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:27:57.595266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ef3ee-5e52-4440-a7d5-a0a93706d81f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:27:57.596958 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ef3ee-5e52-4440-a7d5-a0a93706d81f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:27:57.616690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-c89ef3ee-5e52-4440-a7d5-a0a93706d81f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:27:57.616690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:27:57.616690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:27:57.616690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:27:57.616690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:27:57.616690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:27:57.616690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:27:57.616690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:27:57.616690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:27:57.616690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:27:57.616690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:27:57.616690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:27:57.616690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:27:57.616690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:27:57.616690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:27:57.618549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:27:57.618549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:27:57.618549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:27:57.618549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:27:57.618549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:27:57.618549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:27:57.618549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:27:57.618549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:27:57.618549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:27:57.618549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:27:57.618549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:27:57.618549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:27:57.618549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:27:57.618549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:27:57.618549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:27:57.618549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:27:57.618549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:27:57.619608 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:27:57.619608 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:27:57.619608 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:27:57.619608 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:27:57.619608 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 606/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:27:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:27:57.626501 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05683a-2673-4595-97a7-e2cda40caf82 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:27:57.705130 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:27:57.761411 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05683a-2673-4595-97a7-e2cda40caf82 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:27:57.820188 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060cb4d-0037-4dc4-a315-ae7f05eb9556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:27:57.821650 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060cb4d-0037-4dc4-a315-ae7f05eb9556 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:27:57.822253 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060cb4d-0037-4dc4-a315-ae7f05eb9556 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:27:57.822558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060cb4d-0037-4dc4-a315-ae7f05eb9556 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:27:57.824013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060cb4d-0037-4dc4-a315-ae7f05eb9556 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:27:57.824686 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060cb4d-0037-4dc4-a315-ae7f05eb9556 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:27:57.827075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060cb4d-0037-4dc4-a315-ae7f05eb9556 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:27:57.841034 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 610/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:27:57.851249 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ef3ee-5e52-4440-a7d5-a0a93706d81f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:27:57.936489 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:27:57.972208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ef3ee-5e52-4440-a7d5-a0a93706d81f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:27:58.015500 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf827b-b27e-4b36-9647-090e8508ec2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:27:58.016588 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf827b-b27e-4b36-9647-090e8508ec2c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:27:58.017032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf827b-b27e-4b36-9647-090e8508ec2c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:27:58.017428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf827b-b27e-4b36-9647-090e8508ec2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:27:58.026482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf827b-b27e-4b36-9647-090e8508ec2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:27:58.028254 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf827b-b27e-4b36-9647-090e8508ec2c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:27:58.045217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-fabf827b-b27e-4b36-9647-090e8508ec2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:27:58.045217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:27:58.045217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:27:58.045217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:27:58.045217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:27:58.045217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:27:58.045217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:27:58.045217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:27:58.045217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:27:58.045217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:27:58.045217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:27:58.045217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:27:58.045217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:27:58.045217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:27:58.045217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:27:58.047127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:27:58.047127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:27:58.047127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:27:58.047127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:27:58.047127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:27:58.047127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:27:58.047127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:27:58.047127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:27:58.047127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:27:58.047127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:27:58.047127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:27:58.047127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:27:58.047127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:27:58.047127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:27:58.047127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:27:58.047127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:27:58.047127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:27:58.049054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:27:58.049054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:27:58.049054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:27:58.049054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:27:58.049054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:27:58.049054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:27:58.049054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:27:58.049054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:27:58.049054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 607/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:27:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:27:58.056274 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060cb4d-0037-4dc4-a315-ae7f05eb9556 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:27:58.185363 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060cb4d-0037-4dc4-a315-ae7f05eb9556 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:27:58.215733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:27:58.233226 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc1e22-a7fb-4f1d-92ea-b922656618d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:27:58.233978 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc1e22-a7fb-4f1d-92ea-b922656618d8 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:27:58.234434 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc1e22-a7fb-4f1d-92ea-b922656618d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:27:58.234812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc1e22-a7fb-4f1d-92ea-b922656618d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:27:58.235741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc1e22-a7fb-4f1d-92ea-b922656618d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:27:58.236189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc1e22-a7fb-4f1d-92ea-b922656618d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:27:58.237815 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc1e22-a7fb-4f1d-92ea-b922656618d8 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:27:58.251379 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 611/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:27:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:27:58.257326 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf827b-b27e-4b36-9647-090e8508ec2c None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:27:58.372203 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf827b-b27e-4b36-9647-090e8508ec2c None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:27:58.413265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:27:58.422109 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ace90-0b51-4f2c-a19e-97a5fcd29e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:27:58.423021 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ace90-0b51-4f2c-a19e-97a5fcd29e5a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:27:58.423411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ace90-0b51-4f2c-a19e-97a5fcd29e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:27:58.423866 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ace90-0b51-4f2c-a19e-97a5fcd29e5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:27:58.426754 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ace90-0b51-4f2c-a19e-97a5fcd29e5a None admin] RBAC: Adding query filter params (role.id=95eccd30e39b4105ac6e8c0d9f973951, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=361be69f1c0a41c18afd206bccf56447) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:27:58.426754 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ace90-0b51-4f2c-a19e-97a5fcd29e5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:27:58.426754 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ace90-0b51-4f2c-a19e-97a5fcd29e5a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:27:58.434421 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 608/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:27:58 2022] GET /identity/v3/role_assignments?user.id=361be69f1c0a41c18afd206bccf56447&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=95eccd30e39b4105ac6e8c0d9f973951 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:00.835669 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5da568-e774-4a45-aa58-2663650168c6 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:00.837072 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5da568-e774-4a45-aa58-2663650168c6 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:00.837072 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5da568-e774-4a45-aa58-2663650168c6 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:00.838599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 612/1220] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:28:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:28:00.843909 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb046c-437c-4df6-98a9-cb4e72f793f8 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:00.844354 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb046c-437c-4df6-98a9-cb4e72f793f8 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:00.844650 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb046c-437c-4df6-98a9-cb4e72f793f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:00.863815 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-8ddb046c-437c-4df6-98a9-cb4e72f793f8 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:28:00.865089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-8ddb046c-437c-4df6-98a9-cb4e72f793f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:28:00.941966 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 609/1221] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:28:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:28:00.967071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5da568-e774-4a45-aa58-2663650168c6 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:01.082651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5da568-e774-4a45-aa58-2663650168c6 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:01.119394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47044ad5-e68d-47c3-905c-45d1ebf538bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:01.120140 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47044ad5-e68d-47c3-905c-45d1ebf538bf None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:01.120431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47044ad5-e68d-47c3-905c-45d1ebf538bf None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:01.120704 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47044ad5-e68d-47c3-905c-45d1ebf538bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:01.131454 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47044ad5-e68d-47c3-905c-45d1ebf538bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:28:01.140696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-47044ad5-e68d-47c3-905c-45d1ebf538bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:28:01.140696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:01.140696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:01.140696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:01.140696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:01.140696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:01.140696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:01.140696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:01.140696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:01.140696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:01.140696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:01.140696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:01.140696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:28:01.140696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:28:01.140696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:28:01.140696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:28:01.141781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:28:01.141781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:28:01.141781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:28:01.141781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:28:01.141781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:01.141781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:01.141781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:01.141781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:01.141781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:01.141781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:01.141781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:01.141781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:01.141781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:01.141781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:01.141781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:01.141781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:01.142678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:01.142678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:01.142678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:01.142678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:01.142678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:28:01.142678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:28:01.142678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:28:01.142678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:28:01.142678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:28:01.142678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:28:01.142678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:28:01.142678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:28:01.142678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 613/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:28:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:01.148776 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb046c-437c-4df6-98a9-cb4e72f793f8 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:01.280613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb046c-437c-4df6-98a9-cb4e72f793f8 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:01.304474 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:01.318970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41eb47-8f3e-4e8c-8023-851c1b11ae6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:01.319802 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41eb47-8f3e-4e8c-8023-851c1b11ae6d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:01.320213 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41eb47-8f3e-4e8c-8023-851c1b11ae6d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:01.320604 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41eb47-8f3e-4e8c-8023-851c1b11ae6d None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:01.321462 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41eb47-8f3e-4e8c-8023-851c1b11ae6d None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:01.321863 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41eb47-8f3e-4e8c-8023-851c1b11ae6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:01.323324 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41eb47-8f3e-4e8c-8023-851c1b11ae6d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:01.331741 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 610/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jun 6 10:28:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:01.337986 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47044ad5-e68d-47c3-905c-45d1ebf538bf None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:01.390204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:01.453520 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47044ad5-e68d-47c3-905c-45d1ebf538bf None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:01.497312 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7a187-9669-4c37-bf7d-fbb66bd8d17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:01.498260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7a187-9669-4c37-bf7d-fbb66bd8d17e None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:01.498659 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7a187-9669-4c37-bf7d-fbb66bd8d17e None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:01.499030 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7a187-9669-4c37-bf7d-fbb66bd8d17e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:01.512644 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7a187-9669-4c37-bf7d-fbb66bd8d17e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:01.514124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7a187-9669-4c37-bf7d-fbb66bd8d17e None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:01.532005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-c3c7a187-9669-4c37-bf7d-fbb66bd8d17e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:28:01.532005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:01.532005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:01.532005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:01.532005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:01.532005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:01.532005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:01.532005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:01.532005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:01.532005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:01.532005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:01.532005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:01.532005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:28:01.532005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:28:01.532005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:28:01.533132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:28:01.533132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:01.533132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:01.533132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:01.533132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:01.533132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:01.533132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:01.533132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:01.533132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:01.533132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:01.533132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:01.533132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:01.533132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:01.533132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:01.533132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:01.533132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:01.533132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:01.534288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:28:01.534288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:28:01.534288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:28:01.534288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:28:01.534288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 614/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:28:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:01.538994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41eb47-8f3e-4e8c-8023-851c1b11ae6d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:01.651239 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41eb47-8f3e-4e8c-8023-851c1b11ae6d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:01.688563 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e619a-a633-4fbb-99d0-dc87bdc50914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:01.689618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e619a-a633-4fbb-99d0-dc87bdc50914 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:01.690059 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e619a-a633-4fbb-99d0-dc87bdc50914 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:01.690499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e619a-a633-4fbb-99d0-dc87bdc50914 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:01.691388 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e619a-a633-4fbb-99d0-dc87bdc50914 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:01.691889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e619a-a633-4fbb-99d0-dc87bdc50914 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:01.693423 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e619a-a633-4fbb-99d0-dc87bdc50914 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:01.707111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 611/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:01.713566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7a187-9669-4c37-bf7d-fbb66bd8d17e None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:01.817107 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:01.825993 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7a187-9669-4c37-bf7d-fbb66bd8d17e None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:01.861259 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce20259-d50a-4352-b34a-e2355a47ffe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:01.862228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce20259-d50a-4352-b34a-e2355a47ffe3 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:01.862684 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce20259-d50a-4352-b34a-e2355a47ffe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:01.863052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce20259-d50a-4352-b34a-e2355a47ffe3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:01.872665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce20259-d50a-4352-b34a-e2355a47ffe3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:01.874329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce20259-d50a-4352-b34a-e2355a47ffe3 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:01.891200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-cce20259-d50a-4352-b34a-e2355a47ffe3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 10:28:01.891200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:01.891200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:01.891200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:01.891200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:01.891200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:01.891200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:01.891200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:01.891200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:01.891200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:01.891200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:01.891200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:01.891200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:28:01.891200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:28:01.891200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:28:01.891200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:28:01.892523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:01.892523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:01.892523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:28:01.892523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:28:01.892523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:28:01.892523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:28:01.892523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:01.892523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:01.892523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:01.892523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:01.892523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:01.892523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:01.892523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:01.892523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:01.892523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:01.892523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:01.892523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:01.892523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:01.893748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:01.893748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:01.893748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:28:01.893748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:28:01.893748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:28:01.893748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:28:01.893748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:28:01.893748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:28:01.893748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 10:28:01.893748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:28:01.895104 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 615/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:28:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:01.901488 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e619a-a633-4fbb-99d0-dc87bdc50914 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:01.956889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:02.025986 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e619a-a633-4fbb-99d0-dc87bdc50914 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:02.072049 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f859171-3be4-4ea2-9223-2f4bb18323d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:02.072756 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f859171-3be4-4ea2-9223-2f4bb18323d2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:02.073041 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f859171-3be4-4ea2-9223-2f4bb18323d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:02.073317 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f859171-3be4-4ea2-9223-2f4bb18323d2 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:02.074368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f859171-3be4-4ea2-9223-2f4bb18323d2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:02.074684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f859171-3be4-4ea2-9223-2f4bb18323d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:02.076293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f859171-3be4-4ea2-9223-2f4bb18323d2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:02.099176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 612/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Jun 6 10:28:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:28:02.106148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce20259-d50a-4352-b34a-e2355a47ffe3 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:02.231078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce20259-d50a-4352-b34a-e2355a47ffe3 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:02.268130 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52a3e1-d0c3-4a50-b7c4-2c07aca3cb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:02.268919 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52a3e1-d0c3-4a50-b7c4-2c07aca3cb1d None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:02.269206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52a3e1-d0c3-4a50-b7c4-2c07aca3cb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:02.269496 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52a3e1-d0c3-4a50-b7c4-2c07aca3cb1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:02.281758 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52a3e1-d0c3-4a50-b7c4-2c07aca3cb1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:02.283302 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52a3e1-d0c3-4a50-b7c4-2c07aca3cb1d None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:02.301139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-ee52a3e1-d0c3-4a50-b7c4-2c07aca3cb1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:28:02.301139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:02.301139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:02.301139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:02.301139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:02.301139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:02.301139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:02.301139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:02.301139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:02.301139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:02.301139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:02.301139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:02.301139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:28:02.301139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:28:02.301139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:28:02.302289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:28:02.302289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:02.302289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:02.302289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:02.302289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:02.302289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:02.302289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:02.302289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:02.302289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:02.302289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:02.302289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:02.302289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:02.302289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:02.302289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:02.302289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:02.302289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:02.302289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:02.303526 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:28:02.303526 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:28:02.303526 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:28:02.303526 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:28:02.303526 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 616/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:28:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:28:02.308119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f859171-3be4-4ea2-9223-2f4bb18323d2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:02.418513 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f859171-3be4-4ea2-9223-2f4bb18323d2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:02.422702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:02.457567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d6c6e-44f0-41d0-b507-f839c19609a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:02.458414 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d6c6e-44f0-41d0-b507-f839c19609a0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:02.458788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d6c6e-44f0-41d0-b507-f839c19609a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:02.459068 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d6c6e-44f0-41d0-b507-f839c19609a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:02.459989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d6c6e-44f0-41d0-b507-f839c19609a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:02.460301 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d6c6e-44f0-41d0-b507-f839c19609a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:02.462657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d6c6e-44f0-41d0-b507-f839c19609a0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:02.474954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 613/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:02.480779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52a3e1-d0c3-4a50-b7c4-2c07aca3cb1d None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:02.491622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:02.606338 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52a3e1-d0c3-4a50-b7c4-2c07aca3cb1d None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:02.642995 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70ac7b-456b-403c-b804-cd2375720c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:02.643810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70ac7b-456b-403c-b804-cd2375720c71 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:02.644102 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70ac7b-456b-403c-b804-cd2375720c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:02.644381 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70ac7b-456b-403c-b804-cd2375720c71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:02.654276 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70ac7b-456b-403c-b804-cd2375720c71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:02.655703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70ac7b-456b-403c-b804-cd2375720c71 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:02.671337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-9a70ac7b-456b-403c-b804-cd2375720c71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:28:02.671337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:02.671337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:02.671337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:02.671337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:02.671337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:02.671337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:02.671337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:02.671337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:02.671337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:02.671337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:02.671337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:02.671337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:28:02.671337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:28:02.671337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:28:02.672553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:28:02.672553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:02.672553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:02.672553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:02.672553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:02.672553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:02.672553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:02.672553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:02.672553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:02.672553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:02.672553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:02.672553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:02.672553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:02.672553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:02.672553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:02.672553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:02.672553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:02.679165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:28:02.679165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:28:02.679165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:28:02.679165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:28:02.679165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:28:02.679165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:28:02.679165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:28:02.679165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:28:02.679165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 617/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:28:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:02.679807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d6c6e-44f0-41d0-b507-f839c19609a0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:02.787540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d6c6e-44f0-41d0-b507-f839c19609a0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:02.824815 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80175cf4-542a-4c86-975b-b01d8793424d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:02.825536 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80175cf4-542a-4c86-975b-b01d8793424d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:02.825895 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80175cf4-542a-4c86-975b-b01d8793424d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:02.826188 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80175cf4-542a-4c86-975b-b01d8793424d None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:02.827026 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80175cf4-542a-4c86-975b-b01d8793424d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:02.827358 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80175cf4-542a-4c86-975b-b01d8793424d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:02.828988 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80175cf4-542a-4c86-975b-b01d8793424d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:02.843131 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 614/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:28:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:02.849563 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70ac7b-456b-403c-b804-cd2375720c71 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:02.920764 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:02.959577 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70ac7b-456b-403c-b804-cd2375720c71 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:02.999884 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9740b1-c04e-411b-83b4-c705d9aa915e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:03.001069 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9740b1-c04e-411b-83b4-c705d9aa915e None admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:03.001462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9740b1-c04e-411b-83b4-c705d9aa915e None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:03.001890 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9740b1-c04e-411b-83b4-c705d9aa915e None admin] PATH_INFO: `/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/361be69f1c0a41c18afd206bccf56447/roles/95eccd30e39b4105ac6e8c0d9f973951` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:03.018802 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9740b1-c04e-411b-83b4-c705d9aa915e None admin] RBAC: Authorizing `identity:create_grant(project_id=1f082d7ed3694fd9998623f3d4a5fea9, user_id=361be69f1c0a41c18afd206bccf56447, role_id=95eccd30e39b4105ac6e8c0d9f973951)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:03.020601 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9740b1-c04e-411b-83b4-c705d9aa915e None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:03.035013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 618/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:28:02 2022] PUT /identity/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/361be69f1c0a41c18afd206bccf56447/roles/95eccd30e39b4105ac6e8c0d9f973951 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:28:05.346598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8e233-d898-46c0-8b5f-a02e463038f1 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:05.347147 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8e233-d898-46c0-8b5f-a02e463038f1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:05.347147 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8e233-d898-46c0-8b5f-a02e463038f1 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:05.348538 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 615/1233] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:28:05.354244 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74502103-f7a3-4ef6-bab4-7b1fa985ae27 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:05.354681 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74502103-f7a3-4ef6-bab4-7b1fa985ae27 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:05.354814 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74502103-f7a3-4ef6-bab4-7b1fa985ae27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:05.371777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-74502103-f7a3-4ef6-bab4-7b1fa985ae27 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:28:05.373001 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-74502103-f7a3-4ef6-bab4-7b1fa985ae27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:28:05.449408 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 619/1234] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:28:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:28:05.473854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8e233-d898-46c0-8b5f-a02e463038f1 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:05.502856 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:05.609836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8e233-d898-46c0-8b5f-a02e463038f1 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:05.653779 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ab470-b6cc-4be1-bcde-1319d952f716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:05.654586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ab470-b6cc-4be1-bcde-1319d952f716 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:05.654865 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ab470-b6cc-4be1-bcde-1319d952f716 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:05.655137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ab470-b6cc-4be1-bcde-1319d952f716 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:05.665928 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ab470-b6cc-4be1-bcde-1319d952f716 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:28:05.674402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-054ab470-b6cc-4be1-bcde-1319d952f716 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:28:05.674402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:05.674402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:05.674402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:05.674402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:05.674402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:05.674402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:05.674402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:05.674402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:05.674402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:05.674402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:05.674402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:05.674402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:28:05.674402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:28:05.674402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:28:05.674402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:28:05.675552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:28:05.675552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:28:05.675552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:28:05.675552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:28:05.675552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:05.675552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:05.675552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:05.675552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:05.675552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:05.675552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:05.675552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:05.675552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:05.675552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:05.675552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:05.675552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:05.675552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:05.676475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:05.676475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:05.676475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:05.676475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:05.676475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:28:05.676475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:28:05.676475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:28:05.676475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:28:05.676475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:28:05.676475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:28:05.676475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 10:28:05.676475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:28:05.676475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 616/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jun 6 10:28:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:05.682619 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74502103-f7a3-4ef6-bab4-7b1fa985ae27 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:05.736813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:05.794669 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74502103-f7a3-4ef6-bab4-7b1fa985ae27 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:05.830101 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066dc20-6961-4cf3-8d10-f59985a8772b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:05.830785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066dc20-6961-4cf3-8d10-f59985a8772b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:05.831070 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066dc20-6961-4cf3-8d10-f59985a8772b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:05.831346 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066dc20-6961-4cf3-8d10-f59985a8772b None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:05.832121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9066dc20-6961-4cf3-8d10-f59985a8772b None admin] RBAC: Adding query filter params (name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:05.832436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9066dc20-6961-4cf3-8d10-f59985a8772b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:05.833835 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9066dc20-6961-4cf3-8d10-f59985a8772b None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:05.845306 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 620/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jun 6 10:28:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:28:05.852994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ab470-b6cc-4be1-bcde-1319d952f716 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:05.945297 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:05.965970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ab470-b6cc-4be1-bcde-1319d952f716 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:06.006771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369fe4d3-390a-4319-b400-1a0cba82a7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:06.007641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fe4d3-390a-4319-b400-1a0cba82a7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:06.008021 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fe4d3-390a-4319-b400-1a0cba82a7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:06.008298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fe4d3-390a-4319-b400-1a0cba82a7c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:06.021018 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369fe4d3-390a-4319-b400-1a0cba82a7c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:06.022670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369fe4d3-390a-4319-b400-1a0cba82a7c2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:06.042894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-369fe4d3-390a-4319-b400-1a0cba82a7c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:28:06.042894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:06.042894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:06.042894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:06.042894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:06.042894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:06.042894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:06.042894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:06.042894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:06.042894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:06.042894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:06.042894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:06.042894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:28:06.042894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:28:06.042894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:28:06.043978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:28:06.043978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:06.043978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:06.043978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:06.043978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:06.043978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:06.043978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:06.043978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:06.043978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:06.043978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:06.043978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:06.043978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:06.043978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:06.043978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:06.043978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:06.043978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:06.043978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:06.044888 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:28:06.044888 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:28:06.044888 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:28:06.044888 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:28:06.044888 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 617/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:28:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:06.050554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066dc20-6961-4cf3-8d10-f59985a8772b None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:06.157974 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066dc20-6961-4cf3-8d10-f59985a8772b None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:06.169060 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:06.198228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6bc33f-2c47-4fe7-8228-4881e46c5434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:06.198811 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bc33f-2c47-4fe7-8228-4881e46c5434 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:06.199096 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bc33f-2c47-4fe7-8228-4881e46c5434 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:06.199372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bc33f-2c47-4fe7-8228-4881e46c5434 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:06.200129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6bc33f-2c47-4fe7-8228-4881e46c5434 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:06.200566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6bc33f-2c47-4fe7-8228-4881e46c5434 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:06.202132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6bc33f-2c47-4fe7-8228-4881e46c5434 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:06.217310 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 621/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:06.223503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369fe4d3-390a-4319-b400-1a0cba82a7c2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:06.341115 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369fe4d3-390a-4319-b400-1a0cba82a7c2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:06.383905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fec1a1-306a-4170-8012-c8674ef1ef98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:06.384938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fec1a1-306a-4170-8012-c8674ef1ef98 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:06.385410 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fec1a1-306a-4170-8012-c8674ef1ef98 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:06.385840 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fec1a1-306a-4170-8012-c8674ef1ef98 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:06.389667 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:06.396624 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fec1a1-306a-4170-8012-c8674ef1ef98 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:06.399754 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fec1a1-306a-4170-8012-c8674ef1ef98 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:06.416459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-65fec1a1-306a-4170-8012-c8674ef1ef98 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 10:28:06.416459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:06.416459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:06.416459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:06.416459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:06.416459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:06.416459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:06.416459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:06.416459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:06.416459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:06.416459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:06.416459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:06.416459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:28:06.416459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:28:06.416459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:28:06.416459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:28:06.417503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:06.417503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:06.417503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:28:06.417503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:28:06.417503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:28:06.417503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:28:06.417503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:06.417503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:06.417503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:06.417503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:06.417503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:06.417503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:06.417503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:06.417503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:06.417503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:06.417503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:06.417503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:06.417503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:06.419020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:06.419020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:06.419020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:28:06.419020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:28:06.419020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:28:06.419020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:28:06.419020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:28:06.419020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:28:06.419020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 10:28:06.419020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:28:06.419020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 618/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:28:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:06.423742 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6bc33f-2c47-4fe7-8228-4881e46c5434 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:06.545154 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6bc33f-2c47-4fe7-8228-4881e46c5434 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:06.587548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b48a25-b3e6-4bc8-91e5-ce2a509d3d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:06.588247 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b48a25-b3e6-4bc8-91e5-ce2a509d3d42 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:06.588527 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b48a25-b3e6-4bc8-91e5-ce2a509d3d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:06.588798 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b48a25-b3e6-4bc8-91e5-ce2a509d3d42 None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:06.589628 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b48a25-b3e6-4bc8-91e5-ce2a509d3d42 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:06.590258 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b48a25-b3e6-4bc8-91e5-ce2a509d3d42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:06.591800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b48a25-b3e6-4bc8-91e5-ce2a509d3d42 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:06.611436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 622/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Jun 6 10:28:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:06.617882 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fec1a1-306a-4170-8012-c8674ef1ef98 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:06.729026 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fec1a1-306a-4170-8012-c8674ef1ef98 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:06.765981 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26ffed-ec3d-451a-9e9e-08ac3f5508b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:06.766635 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26ffed-ec3d-451a-9e9e-08ac3f5508b0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:06.766918 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26ffed-ec3d-451a-9e9e-08ac3f5508b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:06.767193 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26ffed-ec3d-451a-9e9e-08ac3f5508b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:06.838136 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:06.846600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26ffed-ec3d-451a-9e9e-08ac3f5508b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:06.848369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26ffed-ec3d-451a-9e9e-08ac3f5508b0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:06.964480 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 619/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:28:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 06 10:28:06.971239 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b48a25-b3e6-4bc8-91e5-ce2a509d3d42 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:07.002344 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:07.083232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b48a25-b3e6-4bc8-91e5-ce2a509d3d42 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:07.123910 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ef7ec-e00d-4eaf-b88d-52183b5d86da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:07.124679 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ef7ec-e00d-4eaf-b88d-52183b5d86da None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:07.125002 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ef7ec-e00d-4eaf-b88d-52183b5d86da None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:07.125327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ef7ec-e00d-4eaf-b88d-52183b5d86da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:07.136684 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8ef7ec-e00d-4eaf-b88d-52183b5d86da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:07.138322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8ef7ec-e00d-4eaf-b88d-52183b5d86da None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:07.157776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-9a8ef7ec-e00d-4eaf-b88d-52183b5d86da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:28:07.157776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:07.157776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:07.157776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:07.157776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:07.157776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:07.157776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:07.157776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:07.157776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:07.157776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:07.157776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:07.157776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:07.157776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:28:07.157776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:28:07.157776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:28:07.158833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:28:07.158833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:07.158833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:07.158833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:07.158833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:07.158833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:07.158833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:07.158833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:07.158833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:07.158833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:07.158833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:07.158833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:07.158833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:07.158833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:07.158833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:07.158833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:07.158833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:07.159743 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:28:07.159743 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:28:07.159743 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:28:07.159743 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:28:07.159743 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 623/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:28:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:07.166610 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26ffed-ec3d-451a-9e9e-08ac3f5508b0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:07.291066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:07.297691 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26ffed-ec3d-451a-9e9e-08ac3f5508b0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:07.333504 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b654544-9c6e-4b7d-972d-5f692b338934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:07.334307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b654544-9c6e-4b7d-972d-5f692b338934 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:07.334439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b654544-9c6e-4b7d-972d-5f692b338934 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:07.334706 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b654544-9c6e-4b7d-972d-5f692b338934 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:07.335524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b654544-9c6e-4b7d-972d-5f692b338934 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:07.335917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b654544-9c6e-4b7d-972d-5f692b338934 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:07.337489 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b654544-9c6e-4b7d-972d-5f692b338934 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:07.349698 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 620/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:28:07.356749 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ef7ec-e00d-4eaf-b88d-52183b5d86da None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:07.443699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:07.466493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ef7ec-e00d-4eaf-b88d-52183b5d86da None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:07.502425 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ff68b-58b4-4bd8-92fb-50e1c0467b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:07.503214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ff68b-58b4-4bd8-92fb-50e1c0467b85 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:07.503498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ff68b-58b4-4bd8-92fb-50e1c0467b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:07.503848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ff68b-58b4-4bd8-92fb-50e1c0467b85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:07.512401 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ff68b-58b4-4bd8-92fb-50e1c0467b85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:07.514302 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ff68b-58b4-4bd8-92fb-50e1c0467b85 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:07.530685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-d75ff68b-58b4-4bd8-92fb-50e1c0467b85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:28:07.530685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:07.530685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:07.530685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:07.530685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:07.530685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:07.530685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:07.530685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:07.530685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:07.530685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:07.530685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:07.530685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:07.530685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:28:07.530685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:28:07.530685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:28:07.531755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:28:07.531755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:07.531755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:07.531755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:07.531755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:07.531755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:07.531755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:07.531755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:07.531755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:07.531755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:07.531755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:07.531755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:07.531755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:07.531755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:07.531755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:07.531755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:07.531755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:07.533339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:28:07.533339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:28:07.533339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:28:07.533339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:28:07.533339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:28:07.533339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:28:07.533339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:28:07.533339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:28:07.533339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 624/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:28:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 10:28:07.538108 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b654544-9c6e-4b7d-972d-5f692b338934 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:07.649758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b654544-9c6e-4b7d-972d-5f692b338934 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:07.688185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099dd3c-bc55-43df-b594-235e1cdf55a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:07.688845 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099dd3c-bc55-43df-b594-235e1cdf55a6 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:07.689126 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099dd3c-bc55-43df-b594-235e1cdf55a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:07.689405 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099dd3c-bc55-43df-b594-235e1cdf55a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:07.690188 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099dd3c-bc55-43df-b594-235e1cdf55a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:07.690490 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099dd3c-bc55-43df-b594-235e1cdf55a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:07.691944 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099dd3c-bc55-43df-b594-235e1cdf55a6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:07.705434 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 621/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:28:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:07.712540 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ff68b-58b4-4bd8-92fb-50e1c0467b85 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:07.825676 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ff68b-58b4-4bd8-92fb-50e1c0467b85 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:07.863596 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c70041-a47c-4934-82c3-c9ff216ba57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:07.864366 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c70041-a47c-4934-82c3-c9ff216ba57b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:07.864754 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c70041-a47c-4934-82c3-c9ff216ba57b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:07.865072 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c70041-a47c-4934-82c3-c9ff216ba57b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:07.866318 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c70041-a47c-4934-82c3-c9ff216ba57b None admin] RBAC: Adding query filter params (role.id=95eccd30e39b4105ac6e8c0d9f973951, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=361be69f1c0a41c18afd206bccf56447) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:07.866691 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c70041-a47c-4934-82c3-c9ff216ba57b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:07.868952 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c70041-a47c-4934-82c3-c9ff216ba57b None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:07.875533 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 625/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:28:07 2022] GET /identity/v3/role_assignments?user.id=361be69f1c0a41c18afd206bccf56447&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=95eccd30e39b4105ac6e8c0d9f973951 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:10.294657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66c985-6d7a-46de-8d0e-1066adefe158 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:10.295236 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66c985-6d7a-46de-8d0e-1066adefe158 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:10.295236 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66c985-6d7a-46de-8d0e-1066adefe158 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:10.296607 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 622/1247] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:28:10.302363 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8d04a-37b2-4b1b-9df7-8cb73761e8aa None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:10.302792 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8d04a-37b2-4b1b-9df7-8cb73761e8aa None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:10.302897 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8d04a-37b2-4b1b-9df7-8cb73761e8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:10.308506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:10.323810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-5ff8d04a-37b2-4b1b-9df7-8cb73761e8aa None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:28:10.325005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-5ff8d04a-37b2-4b1b-9df7-8cb73761e8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:28:10.397172 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 626/1248] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:28:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:28:10.421938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66c985-6d7a-46de-8d0e-1066adefe158 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:10.440943 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:10.534690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66c985-6d7a-46de-8d0e-1066adefe158 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:10.572936 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb365c6-6a8d-48fe-9302-3bca0407754d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:10.573748 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb365c6-6a8d-48fe-9302-3bca0407754d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:10.574099 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb365c6-6a8d-48fe-9302-3bca0407754d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:10.574398 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb365c6-6a8d-48fe-9302-3bca0407754d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:10.587197 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb365c6-6a8d-48fe-9302-3bca0407754d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:28:10.595586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-6bb365c6-6a8d-48fe-9302-3bca0407754d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:28:10.595586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:10.595586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:10.595586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:10.595586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:10.595586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:10.595586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:10.595586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:10.595586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:10.595586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:10.595586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:10.595586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:10.595586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:28:10.595586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:28:10.595586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:28:10.595586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:28:10.596808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:28:10.596808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:28:10.596808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:28:10.596808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:28:10.596808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:10.596808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:10.596808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:10.596808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:10.596808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:10.596808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:10.596808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:10.596808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:10.596808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:10.596808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:10.596808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:10.596808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:10.597854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:10.597854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:10.597854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:10.597854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:10.597854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:28:10.597854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:28:10.597854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:28:10.597854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:28:10.597854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:28:10.597854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:28:10.597854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:28:10.597854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:28:10.597854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 623/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:28:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:10.603639 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8d04a-37b2-4b1b-9df7-8cb73761e8aa None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:10.716789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8d04a-37b2-4b1b-9df7-8cb73761e8aa None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:10.755017 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:10.760724 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47935026-fcb6-4a61-aa74-4ffc6b20dad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:10.761494 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47935026-fcb6-4a61-aa74-4ffc6b20dad6 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:10.761799 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47935026-fcb6-4a61-aa74-4ffc6b20dad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:10.762091 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47935026-fcb6-4a61-aa74-4ffc6b20dad6 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:10.762883 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47935026-fcb6-4a61-aa74-4ffc6b20dad6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:10.763187 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47935026-fcb6-4a61-aa74-4ffc6b20dad6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:10.764643 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47935026-fcb6-4a61-aa74-4ffc6b20dad6 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:10.774641 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 627/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:28:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:10.780576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb365c6-6a8d-48fe-9302-3bca0407754d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:10.873564 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:10.897705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb365c6-6a8d-48fe-9302-3bca0407754d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:10.935447 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ece877-84a7-464b-ab40-d6767ef21dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:10.936276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ece877-84a7-464b-ab40-d6767ef21dda None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:10.936555 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ece877-84a7-464b-ab40-d6767ef21dda None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:10.936895 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ece877-84a7-464b-ab40-d6767ef21dda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:10.949491 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ece877-84a7-464b-ab40-d6767ef21dda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:10.951263 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ece877-84a7-464b-ab40-d6767ef21dda None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:10.969851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-18ece877-84a7-464b-ab40-d6767ef21dda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:28:10.969851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:10.969851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:10.969851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:10.969851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:10.969851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:10.969851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:10.969851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:10.969851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:10.969851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:10.969851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:10.969851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:10.969851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:28:10.969851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:28:10.969851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:28:10.971003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:28:10.971003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:10.971003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:10.971003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:10.971003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:10.971003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:10.971003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:10.971003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:10.971003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:10.971003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:10.971003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:10.971003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:10.971003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:10.971003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:10.971003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:10.971003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:10.971003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:10.971992 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:28:10.971992 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:28:10.971992 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:28:10.971992 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:28:10.971992 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 624/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:28:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:10.978149 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47935026-fcb6-4a61-aa74-4ffc6b20dad6 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:11.085110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47935026-fcb6-4a61-aa74-4ffc6b20dad6 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:11.128290 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0ae23-a990-4454-b5b6-77e6f7dd29e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:11.129273 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0ae23-a990-4454-b5b6-77e6f7dd29e5 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:11.129559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0ae23-a990-4454-b5b6-77e6f7dd29e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:11.129977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0ae23-a990-4454-b5b6-77e6f7dd29e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:11.130946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0ae23-a990-4454-b5b6-77e6f7dd29e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:11.131274 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0ae23-a990-4454-b5b6-77e6f7dd29e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:11.133508 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0ae23-a990-4454-b5b6-77e6f7dd29e5 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:11.147610 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 628/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:28:11.154241 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ece877-84a7-464b-ab40-d6767ef21dda None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:11.268724 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ece877-84a7-464b-ab40-d6767ef21dda None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:11.298417 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:11.308003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca8de79-8d70-43ab-a01c-06cb2dba030b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:11.308885 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca8de79-8d70-43ab-a01c-06cb2dba030b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:11.309196 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca8de79-8d70-43ab-a01c-06cb2dba030b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:11.309493 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca8de79-8d70-43ab-a01c-06cb2dba030b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:11.318054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca8de79-8d70-43ab-a01c-06cb2dba030b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:11.319516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca8de79-8d70-43ab-a01c-06cb2dba030b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:11.334560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-0ca8de79-8d70-43ab-a01c-06cb2dba030b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 10:28:11.334560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:11.334560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:11.334560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:11.334560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:11.334560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:11.334560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:11.334560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:11.334560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:11.334560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:11.334560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:11.334560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:11.334560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:28:11.334560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:28:11.334560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:28:11.334560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:28:11.335994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:11.335994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:11.335994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:28:11.335994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:28:11.335994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:28:11.335994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:28:11.335994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:11.335994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:11.335994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:11.335994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:11.335994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:11.335994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:11.335994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:11.335994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:11.335994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:11.335994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:11.335994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:11.335994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:11.337657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:11.337657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:11.337657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:28:11.337657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:28:11.337657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:28:11.337657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:28:11.337657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:28:11.337657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:28:11.337657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 10:28:11.337657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:28:11.341265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 625/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:28:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:11.341889 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0ae23-a990-4454-b5b6-77e6f7dd29e5 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:11.404807 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:11.456919 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0ae23-a990-4454-b5b6-77e6f7dd29e5 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:11.498904 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485b54c-6afb-4eb4-a38b-6c5d1a6fef1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:11.499932 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485b54c-6afb-4eb4-a38b-6c5d1a6fef1e None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:11.500216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485b54c-6afb-4eb4-a38b-6c5d1a6fef1e None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:11.500491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485b54c-6afb-4eb4-a38b-6c5d1a6fef1e None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:11.501451 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485b54c-6afb-4eb4-a38b-6c5d1a6fef1e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:11.501774 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485b54c-6afb-4eb4-a38b-6c5d1a6fef1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:11.503500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485b54c-6afb-4eb4-a38b-6c5d1a6fef1e None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:11.522504 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 629/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Jun 6 10:28:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:11.529453 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca8de79-8d70-43ab-a01c-06cb2dba030b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:11.640166 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca8de79-8d70-43ab-a01c-06cb2dba030b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:11.678964 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69b6c3-72bb-4dbc-a378-15bf7895e998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:11.679816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69b6c3-72bb-4dbc-a378-15bf7895e998 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:11.680138 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69b6c3-72bb-4dbc-a378-15bf7895e998 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:11.680438 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69b6c3-72bb-4dbc-a378-15bf7895e998 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:11.737085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:11.751603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69b6c3-72bb-4dbc-a378-15bf7895e998 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:11.753196 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69b6c3-72bb-4dbc-a378-15bf7895e998 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:11.859088 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 626/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:28:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 10:28:11.864734 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485b54c-6afb-4eb4-a38b-6c5d1a6fef1e None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:11.975403 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485b54c-6afb-4eb4-a38b-6c5d1a6fef1e None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:11.990728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:12.018901 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07af99c-f599-4577-b63a-7001ac178cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:12.019983 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07af99c-f599-4577-b63a-7001ac178cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:12.020265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07af99c-f599-4577-b63a-7001ac178cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:12.020563 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07af99c-f599-4577-b63a-7001ac178cf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:12.033544 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07af99c-f599-4577-b63a-7001ac178cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:12.035127 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07af99c-f599-4577-b63a-7001ac178cf3 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:12.057124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-d07af99c-f599-4577-b63a-7001ac178cf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:28:12.057124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:12.057124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:12.057124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:12.057124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:12.057124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:12.057124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:12.057124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:12.057124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:12.057124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:12.057124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:12.057124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:12.057124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:28:12.057124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:28:12.057124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:28:12.058227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:28:12.058227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:12.058227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:12.058227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:12.058227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:12.058227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:12.058227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:12.058227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:12.058227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:12.058227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:12.058227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:12.058227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:12.058227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:12.058227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:12.058227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:12.058227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:12.058227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:12.059340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:28:12.059340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:28:12.059340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:28:12.059340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:28:12.059340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 630/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:28:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:12.064837 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69b6c3-72bb-4dbc-a378-15bf7895e998 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:12.159186 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:12.172774 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69b6c3-72bb-4dbc-a378-15bf7895e998 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:12.215893 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0fdd5-3a31-4adc-965e-8bdb9050cf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:12.217044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0fdd5-3a31-4adc-965e-8bdb9050cf9b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:12.217632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0fdd5-3a31-4adc-965e-8bdb9050cf9b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:12.218036 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0fdd5-3a31-4adc-965e-8bdb9050cf9b None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:12.218945 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0fdd5-3a31-4adc-965e-8bdb9050cf9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:12.219247 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0fdd5-3a31-4adc-965e-8bdb9050cf9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:12.221146 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0fdd5-3a31-4adc-965e-8bdb9050cf9b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:12.238086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 627/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:12.244999 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07af99c-f599-4577-b63a-7001ac178cf3 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:12.358656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07af99c-f599-4577-b63a-7001ac178cf3 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:12.396106 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28b9e5-f4c4-4359-af51-12546d522268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:12.396980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28b9e5-f4c4-4359-af51-12546d522268 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:12.397269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28b9e5-f4c4-4359-af51-12546d522268 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:12.397597 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28b9e5-f4c4-4359-af51-12546d522268 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:12.405915 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28b9e5-f4c4-4359-af51-12546d522268 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:12.407344 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28b9e5-f4c4-4359-af51-12546d522268 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:12.422720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-8a28b9e5-f4c4-4359-af51-12546d522268 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:28:12.422720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:12.422720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:12.422720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:12.422720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:12.422720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:12.422720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:12.422720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:12.422720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:12.422720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:12.422720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:12.422720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:12.422720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:28:12.422720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:28:12.422720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:28:12.423892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:28:12.423892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:12.423892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:12.423892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:12.423892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:12.423892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:12.423892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:12.423892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:12.423892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:12.423892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:12.423892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:12.423892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:12.423892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:12.423892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:12.423892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:12.423892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:12.423892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:12.430693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:28:12.430693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:28:12.430693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:28:12.430693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:28:12.430693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:28:12.430693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:28:12.430693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:28:12.430693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:28:12.430693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 631/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:28:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:12.431306 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0fdd5-3a31-4adc-965e-8bdb9050cf9b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:12.548301 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0fdd5-3a31-4adc-965e-8bdb9050cf9b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:12.585421 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f17ecae-a0ee-47ba-b6a8-41284d39622e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:12.586664 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17ecae-a0ee-47ba-b6a8-41284d39622e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:12.586944 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17ecae-a0ee-47ba-b6a8-41284d39622e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:12.587217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17ecae-a0ee-47ba-b6a8-41284d39622e None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:12.588153 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f17ecae-a0ee-47ba-b6a8-41284d39622e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:12.588458 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f17ecae-a0ee-47ba-b6a8-41284d39622e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:12.589916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f17ecae-a0ee-47ba-b6a8-41284d39622e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:12.604163 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 628/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:28:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:12.610621 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28b9e5-f4c4-4359-af51-12546d522268 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:12.639036 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:12.720906 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28b9e5-f4c4-4359-af51-12546d522268 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:12.755519 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10deed3b-b7c5-4d3a-8970-658cf7713d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:12.756278 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10deed3b-b7c5-4d3a-8970-658cf7713d31 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:12.756564 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10deed3b-b7c5-4d3a-8970-658cf7713d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:12.756839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10deed3b-b7c5-4d3a-8970-658cf7713d31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:12.758247 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10deed3b-b7c5-4d3a-8970-658cf7713d31 None admin] RBAC: Adding query filter params (role.id=0f929204eb7b4ea8859338ed2d3ff75b, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=361be69f1c0a41c18afd206bccf56447) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:12.758556 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10deed3b-b7c5-4d3a-8970-658cf7713d31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:12.760022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10deed3b-b7c5-4d3a-8970-658cf7713d31 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:12.766059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 632/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:28:12 2022] GET /identity/v3/role_assignments?user.id=361be69f1c0a41c18afd206bccf56447&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=0f929204eb7b4ea8859338ed2d3ff75b => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:15.048197 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404433cb-412b-429f-a288-b48c301c4aaa None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:15.048727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404433cb-412b-429f-a288-b48c301c4aaa None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:15.048727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404433cb-412b-429f-a288-b48c301c4aaa None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:15.050099 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 629/1261] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:28:15.056148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56005892-6229-430d-9acb-b968ec1a02f5 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:15.056580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56005892-6229-430d-9acb-b968ec1a02f5 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:15.056704 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56005892-6229-430d-9acb-b968ec1a02f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:15.072967 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-56005892-6229-430d-9acb-b968ec1a02f5 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:28:15.074121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-56005892-6229-430d-9acb-b968ec1a02f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:28:15.138092 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 633/1262] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:28:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:28:15.160819 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404433cb-412b-429f-a288-b48c301c4aaa None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:15.172036 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:15.278754 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404433cb-412b-429f-a288-b48c301c4aaa None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:15.312655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14662b-a9fa-43e8-ab69-eb86dcc1b01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:15.313366 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14662b-a9fa-43e8-ab69-eb86dcc1b01e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:15.313642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14662b-a9fa-43e8-ab69-eb86dcc1b01e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:15.313910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14662b-a9fa-43e8-ab69-eb86dcc1b01e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:15.323532 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14662b-a9fa-43e8-ab69-eb86dcc1b01e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:28:15.331089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-fe14662b-a9fa-43e8-ab69-eb86dcc1b01e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:28:15.331089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:15.331089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:15.331089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:15.331089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:15.331089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:15.331089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:15.331089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:15.331089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:15.331089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:15.331089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:15.331089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:15.331089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:28:15.331089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:28:15.331089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:28:15.331089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:28:15.332202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:28:15.332202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:28:15.332202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:28:15.332202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:28:15.332202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:15.332202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:15.332202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:15.332202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:15.332202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:15.332202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:15.332202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:15.332202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:15.332202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:15.332202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:15.332202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:15.332202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:15.333086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:15.333086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:15.333086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:15.333086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:15.333086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:28:15.333086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:28:15.333086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:28:15.333086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:28:15.333086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:28:15.333086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:28:15.333086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:28:15.333086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:28:15.333086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 630/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:28:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:15.338886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56005892-6229-430d-9acb-b968ec1a02f5 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:15.357400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:15.436809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56005892-6229-430d-9acb-b968ec1a02f5 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:15.470332 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53d5ef-7286-4fb5-a16a-8b605194e750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:15.470983 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53d5ef-7286-4fb5-a16a-8b605194e750 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:15.471260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53d5ef-7286-4fb5-a16a-8b605194e750 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:15.471534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53d5ef-7286-4fb5-a16a-8b605194e750 None admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:15.472289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff53d5ef-7286-4fb5-a16a-8b605194e750 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:15.472589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff53d5ef-7286-4fb5-a16a-8b605194e750 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:15.473941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff53d5ef-7286-4fb5-a16a-8b605194e750 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:15.482518 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 634/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:28:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:15.488442 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14662b-a9fa-43e8-ab69-eb86dcc1b01e None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:15.553783 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:15.591432 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14662b-a9fa-43e8-ab69-eb86dcc1b01e None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:15.624347 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6a65d-9ef5-4036-8f00-9c42f879477f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:15.625134 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6a65d-9ef5-4036-8f00-9c42f879477f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:15.625414 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6a65d-9ef5-4036-8f00-9c42f879477f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:15.625684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6a65d-9ef5-4036-8f00-9c42f879477f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:15.636329 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6a65d-9ef5-4036-8f00-9c42f879477f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:15.637764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6a65d-9ef5-4036-8f00-9c42f879477f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:15.655214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-25f6a65d-9ef5-4036-8f00-9c42f879477f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:28:15.655214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:15.655214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:15.655214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:15.655214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:15.655214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:15.655214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:15.655214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:15.655214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:15.655214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:15.655214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:15.655214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:15.655214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:28:15.655214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:28:15.655214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:28:15.656275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:28:15.656275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:15.656275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:15.656275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:15.656275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:15.656275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:15.656275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:15.656275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:15.656275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:15.656275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:15.656275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:15.656275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:15.656275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:15.656275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:15.656275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:15.656275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:15.656275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:15.657359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:28:15.657359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:28:15.657359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:28:15.657359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:28:15.657359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 631/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:28:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:15.662225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53d5ef-7286-4fb5-a16a-8b605194e750 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:15.739872 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:15.758670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53d5ef-7286-4fb5-a16a-8b605194e750 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:15.791450 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f38f36-0d06-4096-82ee-9a6a4543de2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:15.792129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f38f36-0d06-4096-82ee-9a6a4543de2b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:15.792411 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f38f36-0d06-4096-82ee-9a6a4543de2b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:15.792687 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f38f36-0d06-4096-82ee-9a6a4543de2b None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:15.793394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f38f36-0d06-4096-82ee-9a6a4543de2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:15.793697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f38f36-0d06-4096-82ee-9a6a4543de2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:15.795048 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f38f36-0d06-4096-82ee-9a6a4543de2b None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:15.806317 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 635/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:15.811892 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6a65d-9ef5-4036-8f00-9c42f879477f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:15.923206 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6a65d-9ef5-4036-8f00-9c42f879477f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:15.942750 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:15.957851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8d88b-edf6-4239-8194-300537fad5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:15.958637 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8d88b-edf6-4239-8194-300537fad5bb None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:15.958916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8d88b-edf6-4239-8194-300537fad5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:15.959184 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8d88b-edf6-4239-8194-300537fad5bb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:15.967508 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8d88b-edf6-4239-8194-300537fad5bb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:15.968957 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8d88b-edf6-4239-8194-300537fad5bb None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:15.983267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-fae8d88b-edf6-4239-8194-300537fad5bb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 10:28:15.983267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:15.983267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:15.983267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:15.983267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:15.983267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:15.983267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:15.983267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:15.983267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:15.983267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:15.983267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:15.983267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:15.983267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:28:15.983267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:28:15.983267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:28:15.983267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:28:15.984367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:15.984367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:15.984367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:28:15.984367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:28:15.984367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:28:15.984367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:28:15.984367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:15.984367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:15.984367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:15.984367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:15.984367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:15.984367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:15.984367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:15.984367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:15.984367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:15.984367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:15.984367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:15.984367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:15.985516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:15.985516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:15.985516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:28:15.985516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:28:15.985516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:28:15.985516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:28:15.985516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:28:15.985516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:28:15.985516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 10:28:15.985516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:28:15.985516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 632/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:28:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:15.990205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f38f36-0d06-4096-82ee-9a6a4543de2b None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:16.087574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f38f36-0d06-4096-82ee-9a6a4543de2b None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:16.120108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39facab-4fef-477f-bc38-cae257fdd927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:16.120758 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39facab-4fef-477f-bc38-cae257fdd927 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:16.121045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39facab-4fef-477f-bc38-cae257fdd927 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:16.121319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39facab-4fef-477f-bc38-cae257fdd927 None admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:16.122142 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39facab-4fef-477f-bc38-cae257fdd927 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:16.122448 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39facab-4fef-477f-bc38-cae257fdd927 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:16.123879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39facab-4fef-477f-bc38-cae257fdd927 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:16.132124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:16.140738 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 636/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Jun 6 10:28:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:16.146616 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8d88b-edf6-4239-8194-300537fad5bb None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:16.257302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8d88b-edf6-4239-8194-300537fad5bb None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:16.289949 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a514cdb8-7adf-4cd5-a98d-cb0e266606ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:16.290721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514cdb8-7adf-4cd5-a98d-cb0e266606ea None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:16.290997 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514cdb8-7adf-4cd5-a98d-cb0e266606ea None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:16.291263 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514cdb8-7adf-4cd5-a98d-cb0e266606ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:16.301168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a514cdb8-7adf-4cd5-a98d-cb0e266606ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:16.302600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a514cdb8-7adf-4cd5-a98d-cb0e266606ea None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:16.319074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-a514cdb8-7adf-4cd5-a98d-cb0e266606ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:28:16.319074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:16.319074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:16.319074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:16.319074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:16.319074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:16.319074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:16.319074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:16.319074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:16.319074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:16.319074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:16.319074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:16.319074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:28:16.319074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:28:16.319074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:28:16.319982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:28:16.319982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:16.319982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:16.319982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:16.319982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:16.319982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:16.319982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:16.319982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:16.319982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:16.319982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:16.319982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:16.319982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:16.319982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:16.319982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:16.319982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:16.319982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:16.319982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:16.320890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:28:16.320890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:28:16.320890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:28:16.320890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:28:16.320890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 633/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:28:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:16.326046 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39facab-4fef-477f-bc38-cae257fdd927 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:16.424775 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39facab-4fef-477f-bc38-cae257fdd927 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:16.457482 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc3b8b-6dd0-4387-b94d-90f9f13f64b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:16.458132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc3b8b-6dd0-4387-b94d-90f9f13f64b9 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:16.458413 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc3b8b-6dd0-4387-b94d-90f9f13f64b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:16.458686 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc3b8b-6dd0-4387-b94d-90f9f13f64b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:16.459399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc3b8b-6dd0-4387-b94d-90f9f13f64b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:16.459705 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc3b8b-6dd0-4387-b94d-90f9f13f64b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:16.461763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc3b8b-6dd0-4387-b94d-90f9f13f64b9 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:16.472538 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 637/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:16.478534 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a514cdb8-7adf-4cd5-a98d-cb0e266606ea None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:16.509639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:16.575949 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a514cdb8-7adf-4cd5-a98d-cb0e266606ea None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:16.608079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9a530-dc06-44df-b78f-c424aee1ad33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:16.608849 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9a530-dc06-44df-b78f-c424aee1ad33 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:16.609126 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9a530-dc06-44df-b78f-c424aee1ad33 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:16.609391 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9a530-dc06-44df-b78f-c424aee1ad33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:16.615753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a9a530-dc06-44df-b78f-c424aee1ad33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:16.617220 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a9a530-dc06-44df-b78f-c424aee1ad33 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:16.630967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-a1a9a530-dc06-44df-b78f-c424aee1ad33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:28:16.630967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:16.630967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:16.630967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:16.630967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:16.630967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:16.630967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:16.630967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:16.630967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:16.630967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:16.630967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:16.630967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:16.630967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:28:16.630967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:28:16.630967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:28:16.632035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:28:16.632035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:16.632035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:16.632035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:16.632035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:16.632035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:16.632035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:16.632035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:16.632035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:16.632035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:16.632035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:16.632035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:16.632035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:16.632035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:16.632035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:16.632035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:16.632035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:16.633497 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:28:16.633497 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:28:16.633497 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:28:16.633497 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:28:16.633497 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:28:16.633497 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:28:16.633497 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:28:16.633497 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:28:16.633497 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 634/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:28:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:16.638423 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc3b8b-6dd0-4387-b94d-90f9f13f64b9 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:16.696267 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:16.737474 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc3b8b-6dd0-4387-b94d-90f9f13f64b9 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:16.770887 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae790f-6ac0-4c54-b8de-aec46aee293b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:16.771471 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae790f-6ac0-4c54-b8de-aec46aee293b None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:16.771756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae790f-6ac0-4c54-b8de-aec46aee293b None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:16.772061 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae790f-6ac0-4c54-b8de-aec46aee293b None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:16.772967 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fae790f-6ac0-4c54-b8de-aec46aee293b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:16.773283 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fae790f-6ac0-4c54-b8de-aec46aee293b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:16.774712 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fae790f-6ac0-4c54-b8de-aec46aee293b None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:16.786303 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 638/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:28:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:16.792199 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9a530-dc06-44df-b78f-c424aee1ad33 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:16.878566 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:16.887884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9a530-dc06-44df-b78f-c424aee1ad33 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:16.921615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f0687-f64d-4171-b258-c3312b161121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:16.922495 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f0687-f64d-4171-b258-c3312b161121 None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:16.922778 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f0687-f64d-4171-b258-c3312b161121 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:16.923052 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f0687-f64d-4171-b258-c3312b161121 None admin] PATH_INFO: `/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/361be69f1c0a41c18afd206bccf56447/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:16.924992 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f0687-f64d-4171-b258-c3312b161121 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f082d7ed3694fd9998623f3d4a5fea9, user_id=361be69f1c0a41c18afd206bccf56447, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:16.926415 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f0687-f64d-4171-b258-c3312b161121 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:16.939828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 635/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jun 6 10:28:16 2022] PUT /identity/v3/projects/1f082d7ed3694fd9998623f3d4a5fea9/users/361be69f1c0a41c18afd206bccf56447/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:28:19.325998 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97cf1fe-7d57-4694-ae16-695a6b279186 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:19.326529 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97cf1fe-7d57-4694-ae16-695a6b279186 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:19.326529 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97cf1fe-7d57-4694-ae16-695a6b279186 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:19.327912 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 639/1274] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:28:19.333308 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa62f4-0559-48fc-beef-9f9853aa488d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:19.333716 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa62f4-0559-48fc-beef-9f9853aa488d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:19.333850 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa62f4-0559-48fc-beef-9f9853aa488d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:19.351277 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-82aa62f4-0559-48fc-beef-9f9853aa488d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:28:19.352477 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-82aa62f4-0559-48fc-beef-9f9853aa488d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:28:19.421284 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 636/1275] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:28:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 10:28:19.445174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97cf1fe-7d57-4694-ae16-695a6b279186 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:19.551254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97cf1fe-7d57-4694-ae16-695a6b279186 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:19.569161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:19.596100 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55297aa-3840-476c-8987-edccbdb7c80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:19.596817 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55297aa-3840-476c-8987-edccbdb7c80e None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:19.597102 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55297aa-3840-476c-8987-edccbdb7c80e None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:19.597376 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55297aa-3840-476c-8987-edccbdb7c80e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:19.613106 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55297aa-3840-476c-8987-edccbdb7c80e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 10:28:19.621624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-c55297aa-3840-476c-8987-edccbdb7c80e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:28:19.621624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:19.621624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:19.621624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:19.621624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:19.621624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:19.621624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:19.621624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:19.621624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:19.621624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:19.621624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:19.621624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:19.621624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 10:28:19.621624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 10:28:19.621624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 10:28:19.621624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 10:28:19.622746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 10:28:19.622746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 10:28:19.622746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 10:28:19.622746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 10:28:19.622746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:19.622746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:19.622746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:19.622746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:19.622746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:19.622746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:19.622746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:19.622746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:19.622746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:19.622746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:19.622746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:19.622746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:19.623964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:19.623964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:19.623964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:19.623964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:19.623964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 10:28:19.623964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 10:28:19.623964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 10:28:19.623964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 10:28:19.623964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 10:28:19.623964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 10:28:19.623964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 10:28:19.623964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:28:19.623964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 640/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jun 6 10:28:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:19.628890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa62f4-0559-48fc-beef-9f9853aa488d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:19.709111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:19.729148 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa62f4-0559-48fc-beef-9f9853aa488d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:19.761981 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f1a4f-0a3d-4c39-8a33-456af3317fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:19.762646 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f1a4f-0a3d-4c39-8a33-456af3317fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:19.762930 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f1a4f-0a3d-4c39-8a33-456af3317fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:19.763207 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f1a4f-0a3d-4c39-8a33-456af3317fd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:19.763976 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f1a4f-0a3d-4c39-8a33-456af3317fd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:19.764285 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f1a4f-0a3d-4c39-8a33-456af3317fd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:19.765648 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f1a4f-0a3d-4c39-8a33-456af3317fd3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:19.774284 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 637/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jun 6 10:28:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:19.780018 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55297aa-3840-476c-8987-edccbdb7c80e None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:19.876958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55297aa-3840-476c-8987-edccbdb7c80e None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:19.909662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed02ab2-e382-4adb-ba22-3e723598d036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:19.910433 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed02ab2-e382-4adb-ba22-3e723598d036 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:19.910707 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed02ab2-e382-4adb-ba22-3e723598d036 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:19.910974 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed02ab2-e382-4adb-ba22-3e723598d036 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:19.921754 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed02ab2-e382-4adb-ba22-3e723598d036 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:19.923226 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed02ab2-e382-4adb-ba22-3e723598d036 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:19.939946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-9ed02ab2-e382-4adb-ba22-3e723598d036 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:28:19.939946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:19.939946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:19.939946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:19.939946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:19.939946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:19.939946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:19.939946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:19.939946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:19.939946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:19.939946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:19.939946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:19.939946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:28:19.939946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:28:19.939946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:28:19.941122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:28:19.941122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:19.941122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:19.941122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:19.941122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:19.941122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:19.941122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:19.941122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:19.941122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:19.941122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:19.941122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:19.941122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:19.941122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:19.941122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:19.941122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:19.941122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:19.941122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:19.942045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:28:19.942045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:28:19.942045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:28:19.942045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:28:19.942045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 641/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:28:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:19.946976 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f1a4f-0a3d-4c39-8a33-456af3317fd3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:20.047411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f1a4f-0a3d-4c39-8a33-456af3317fd3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:20.080566 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae2658-a6cb-44d4-b79c-4b5957473e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:20.081213 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae2658-a6cb-44d4-b79c-4b5957473e66 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:20.081491 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae2658-a6cb-44d4-b79c-4b5957473e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:20.081763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae2658-a6cb-44d4-b79c-4b5957473e66 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:20.082472 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae2658-a6cb-44d4-b79c-4b5957473e66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:20.082772 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae2658-a6cb-44d4-b79c-4b5957473e66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:20.084171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae2658-a6cb-44d4-b79c-4b5957473e66 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:20.095738 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 638/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:28:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:20.102256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed02ab2-e382-4adb-ba22-3e723598d036 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:20.137531 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:20.210145 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed02ab2-e382-4adb-ba22-3e723598d036 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:20.250327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce473cf-de26-4bf0-9d19-a019a5c9de46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:20.251251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce473cf-de26-4bf0-9d19-a019a5c9de46 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:20.251615 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce473cf-de26-4bf0-9d19-a019a5c9de46 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:20.251964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce473cf-de26-4bf0-9d19-a019a5c9de46 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:20.261483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce473cf-de26-4bf0-9d19-a019a5c9de46 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:20.263222 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce473cf-de26-4bf0-9d19-a019a5c9de46 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:20.278672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-5ce473cf-de26-4bf0-9d19-a019a5c9de46 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 10:28:20.278672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:20.278672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:20.278672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:20.278672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:20.278672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:20.278672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:20.278672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:20.278672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:20.278672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:20.278672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:20.278672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:20.278672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 10:28:20.278672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 10:28:20.278672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 10:28:20.278672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 10:28:20.279849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:20.279849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:20.279849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 10:28:20.279849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:28:20.279849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 10:28:20.279849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 10:28:20.279849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:20.279849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:20.279849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:20.279849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:20.279849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:20.279849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:20.279849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:20.279849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:20.279849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:20.279849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:20.279849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:20.279849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:20.282832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:20.282832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:20.282832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 10:28:20.282832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 10:28:20.282832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 10:28:20.282832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 10:28:20.282832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 10:28:20.282832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 10:28:20.282832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 10:28:20.282832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:28:20.282832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 642/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:28:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:20.286338 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae2658-a6cb-44d4-b79c-4b5957473e66 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:20.290933 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:20.385428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae2658-a6cb-44d4-b79c-4b5957473e66 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:20.418872 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d9cfe-4903-424a-a1b9-b776f618c6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:20.419470 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d9cfe-4903-424a-a1b9-b776f618c6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:20.419754 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d9cfe-4903-424a-a1b9-b776f618c6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:20.420063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d9cfe-4903-424a-a1b9-b776f618c6b9 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:20.420899 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d9cfe-4903-424a-a1b9-b776f618c6b9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:20.421206 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d9cfe-4903-424a-a1b9-b776f618c6b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:20.422628 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d9cfe-4903-424a-a1b9-b776f618c6b9 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:20.439581 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 639/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Jun 6 10:28:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:20.445368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce473cf-de26-4bf0-9d19-a019a5c9de46 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:20.534520 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:20.544673 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce473cf-de26-4bf0-9d19-a019a5c9de46 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:20.579742 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519af1b9-0ffd-4340-b563-e8fd4232bb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:20.580412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519af1b9-0ffd-4340-b563-e8fd4232bb1a None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:20.580697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519af1b9-0ffd-4340-b563-e8fd4232bb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:20.580971 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519af1b9-0ffd-4340-b563-e8fd4232bb1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:20.649498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519af1b9-0ffd-4340-b563-e8fd4232bb1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:20.650951 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519af1b9-0ffd-4340-b563-e8fd4232bb1a None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:20.741139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:20.752297 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 643/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jun 6 10:28:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 10:28:20.758604 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d9cfe-4903-424a-a1b9-b776f618c6b9 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:20.816760 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:20.855879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d9cfe-4903-424a-a1b9-b776f618c6b9 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:20.889103 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5cc3ab-9eb0-49ae-b994-09c04b6c78a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:20.889882 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cc3ab-9eb0-49ae-b994-09c04b6c78a6 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:20.890160 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cc3ab-9eb0-49ae-b994-09c04b6c78a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:20.890427 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cc3ab-9eb0-49ae-b994-09c04b6c78a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:20.901130 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5cc3ab-9eb0-49ae-b994-09c04b6c78a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:20.902563 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5cc3ab-9eb0-49ae-b994-09c04b6c78a6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:20.919960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-ca5cc3ab-9eb0-49ae-b994-09c04b6c78a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:28:20.919960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:20.919960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:20.919960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:20.919960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:20.919960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:20.919960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:20.919960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:20.919960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:20.919960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:20.919960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:20.919960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:20.919960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 10:28:20.919960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 10:28:20.919960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 10:28:20.921008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 10:28:20.921008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:20.921008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:20.921008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:20.921008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:20.921008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:20.921008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:20.921008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:20.921008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:20.921008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:20.921008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:20.921008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:20.921008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:20.921008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:20.921008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:20.921008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:20.921008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:20.921933 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 10:28:20.921933 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 10:28:20.921933 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 10:28:20.921933 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:28:20.921933 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 640/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:28:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:20.926824 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519af1b9-0ffd-4340-b563-e8fd4232bb1a None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:21.044384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519af1b9-0ffd-4340-b563-e8fd4232bb1a None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:21.077118 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98100e28-dcaf-4a90-a5fb-b81a7f9a4aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:21.077773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98100e28-dcaf-4a90-a5fb-b81a7f9a4aee None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:21.078055 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98100e28-dcaf-4a90-a5fb-b81a7f9a4aee None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:21.078330 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98100e28-dcaf-4a90-a5fb-b81a7f9a4aee None admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:21.079046 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98100e28-dcaf-4a90-a5fb-b81a7f9a4aee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:21.079350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98100e28-dcaf-4a90-a5fb-b81a7f9a4aee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:21.080742 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98100e28-dcaf-4a90-a5fb-b81a7f9a4aee None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:21.091820 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 644/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jun 6 10:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:21.097378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5cc3ab-9eb0-49ae-b994-09c04b6c78a6 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:21.196364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5cc3ab-9eb0-49ae-b994-09c04b6c78a6 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:21.211162 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:21.231200 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134c8cf-ea4d-476f-b634-5daa64c62c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:21.232021 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134c8cf-ea4d-476f-b634-5daa64c62c7d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:21.232302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134c8cf-ea4d-476f-b634-5daa64c62c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:21.232572 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134c8cf-ea4d-476f-b634-5daa64c62c7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:21.238974 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134c8cf-ea4d-476f-b634-5daa64c62c7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:21.240436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134c8cf-ea4d-476f-b634-5daa64c62c7d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:21.253797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-5134c8cf-ea4d-476f-b634-5daa64c62c7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:28:21.253797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:21.253797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:21.253797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:21.253797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:21.253797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:21.253797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:21.253797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:21.253797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:21.253797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:21.253797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:21.253797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:21.253797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 10:28:21.253797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 10:28:21.253797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 10:28:21.254831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 10:28:21.254831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:21.254831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:21.254831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:21.254831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:21.254831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:21.254831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:21.254831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:21.254831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:21.254831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:21.254831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:21.254831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:21.254831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:21.254831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:21.254831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:21.254831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:21.254831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:21.255736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 10:28:21.255736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 10:28:21.255736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 10:28:21.255736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 10:28:21.255736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 10:28:21.255736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 10:28:21.255736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 10:28:21.255736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jun 06 10:28:21.255736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 641/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jun 6 10:28:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:21.260795 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98100e28-dcaf-4a90-a5fb-b81a7f9a4aee None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:21.314701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:21.364331 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98100e28-dcaf-4a90-a5fb-b81a7f9a4aee None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:21.399060 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a8f03-bdb8-44be-8131-a77a3aac1645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:21.399709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a8f03-bdb8-44be-8131-a77a3aac1645 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:21.400016 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a8f03-bdb8-44be-8131-a77a3aac1645 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:21.400289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a8f03-bdb8-44be-8131-a77a3aac1645 None admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:21.401061 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a8f03-bdb8-44be-8131-a77a3aac1645 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:21.401365 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a8f03-bdb8-44be-8131-a77a3aac1645 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:21.402772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a8f03-bdb8-44be-8131-a77a3aac1645 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:21.414947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 645/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jun 6 10:28:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:28:21.421040 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134c8cf-ea4d-476f-b634-5daa64c62c7d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:21.518306 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134c8cf-ea4d-476f-b634-5daa64c62c7d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:21.550375 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e00607-c0eb-457f-89ef-e9ed6a6637b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:21.551002 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e00607-c0eb-457f-89ef-e9ed6a6637b5 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:21.551281 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e00607-c0eb-457f-89ef-e9ed6a6637b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:21.551553 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e00607-c0eb-457f-89ef-e9ed6a6637b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:21.552395 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e00607-c0eb-457f-89ef-e9ed6a6637b5 None admin] RBAC: Adding query filter params (role.id=0f929204eb7b4ea8859338ed2d3ff75b, scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9, user.id=361be69f1c0a41c18afd206bccf56447) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:21.552700 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e00607-c0eb-457f-89ef-e9ed6a6637b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:21.554858 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e00607-c0eb-457f-89ef-e9ed6a6637b5 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:21.560459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 642/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jun 6 10:28:21 2022] GET /identity/v3/role_assignments?user.id=361be69f1c0a41c18afd206bccf56447&scope.project.id=1f082d7ed3694fd9998623f3d4a5fea9&role.id=0f929204eb7b4ea8859338ed2d3ff75b => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:23.802143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35070f-2367-45bc-8d7d-9a3c5327e38c None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:23.802709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35070f-2367-45bc-8d7d-9a3c5327e38c None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:23.802709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35070f-2367-45bc-8d7d-9a3c5327e38c None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:23.804821 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 646/1288] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:28:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:28:23.809983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bb318-d5c6-495e-8eb1-af396ffbd9f3 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:23.810367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bb318-d5c6-495e-8eb1-af396ffbd9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:23.810540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bb318-d5c6-495e-8eb1-af396ffbd9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:23.828352 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-017bb318-d5c6-495e-8eb1-af396ffbd9f3 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:28:23.829516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-017bb318-d5c6-495e-8eb1-af396ffbd9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:28:23.858283 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:23.894254 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 643/1289] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:28:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:28:23.916866 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35070f-2367-45bc-8d7d-9a3c5327e38c None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:24.017644 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35070f-2367-45bc-8d7d-9a3c5327e38c None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:24.028686 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:24.051544 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4ec51-5919-4a0b-bc67-9ac3f5502d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:24.052399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4ec51-5919-4a0b-bc67-9ac3f5502d25 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:24.052629 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4ec51-5919-4a0b-bc67-9ac3f5502d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:24.052896 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4ec51-5919-4a0b-bc67-9ac3f5502d25 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:24.053574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec4ec51-5919-4a0b-bc67-9ac3f5502d25 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:24.055417 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec4ec51-5919-4a0b-bc67-9ac3f5502d25 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:24.066853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-cec4ec51-5919-4a0b-bc67-9ac3f5502d25 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 10:28:24.066853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:24.066853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:28:24.066853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:28:24.066853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:28:24.066853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:28:24.066853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:28:24.066853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:28:24.066853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 10:28:24.066853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:28:24.066853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:28:24.066853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:28:24.066853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:24.066853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:24.066853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:24.066853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:24.066853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:24.068080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:24.068080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:24.068080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:24.068080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:24.068080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:24.068080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:24.068080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:28:24.068080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:28:24.068080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:28:24.068080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:28:24.068080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:24.068080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:24.068080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:24.068080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:24.068080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:24.068080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:24.069232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:24.069232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:24.069232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:24.069232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:24.069232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:24.069232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:24.069232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:24.069232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:24.069232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:24.069232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:24.069232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:28:24.069232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:28:24.069232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 10:28:24.069232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:28:24.069232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 647/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jun 6 10:28:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:24.074297 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bb318-d5c6-495e-8eb1-af396ffbd9f3 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:24.169498 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bb318-d5c6-495e-8eb1-af396ffbd9f3 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:24.203993 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a44edd8-f226-49c9-9626-2f6322462f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:24.204750 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44edd8-f226-49c9-9626-2f6322462f35 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:24.205093 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44edd8-f226-49c9-9626-2f6322462f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:24.205471 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44edd8-f226-49c9-9626-2f6322462f35 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:24.206402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a44edd8-f226-49c9-9626-2f6322462f35 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:24.206804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a44edd8-f226-49c9-9626-2f6322462f35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:24.208374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a44edd8-f226-49c9-9626-2f6322462f35 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:24.214783 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 644/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Jun 6 10:28:24 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:24.221163 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4ec51-5919-4a0b-bc67-9ac3f5502d25 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:24.333687 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4ec51-5919-4a0b-bc67-9ac3f5502d25 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:24.367067 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3dccc-1a40-4158-b56f-2976e59e52a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:24.367716 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3dccc-1a40-4158-b56f-2976e59e52a3 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:24.368028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3dccc-1a40-4158-b56f-2976e59e52a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:24.368303 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3dccc-1a40-4158-b56f-2976e59e52a3 None admin] PATH_INFO: `/v3/services` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:24.368998 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3dccc-1a40-4158-b56f-2976e59e52a3 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:24.369303 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3dccc-1a40-4158-b56f-2976e59e52a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:24.370686 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3dccc-1a40-4158-b56f-2976e59e52a3 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:24.375077 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 648/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Jun 6 10:28:24 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:28:26.612443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e5f5a-e170-4f8f-aa89-a4fa69924446 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:26.613000 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e5f5a-e170-4f8f-aa89-a4fa69924446 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:26.613000 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e5f5a-e170-4f8f-aa89-a4fa69924446 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:26.614345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 645/1293] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:28:26.619635 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce98eb-ca4c-4321-acf9-6658bb66af47 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:26.620106 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce98eb-ca4c-4321-acf9-6658bb66af47 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:26.620226 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce98eb-ca4c-4321-acf9-6658bb66af47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:26.636709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-7cce98eb-ca4c-4321-acf9-6658bb66af47 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:28:26.637850 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-7cce98eb-ca4c-4321-acf9-6658bb66af47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:28:26.667254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:26.704011 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 649/1294] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:28:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:28:26.727456 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e5f5a-e170-4f8f-aa89-a4fa69924446 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:26.757461 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:26.826307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e5f5a-e170-4f8f-aa89-a4fa69924446 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:26.859514 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4acde8-fd76-4edb-9e51-50b86c40dbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:26.860188 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4acde8-fd76-4edb-9e51-50b86c40dbcb None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:26.860472 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4acde8-fd76-4edb-9e51-50b86c40dbcb None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:26.860747 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4acde8-fd76-4edb-9e51-50b86c40dbcb None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:26.861455 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4acde8-fd76-4edb-9e51-50b86c40dbcb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:26.862822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4acde8-fd76-4edb-9e51-50b86c40dbcb None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:26.876099 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 646/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jun 6 10:28:26 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:28:29.131897 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb92694-733d-4b24-8b85-0391d0a4f6f5 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:29.132478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb92694-733d-4b24-8b85-0391d0a4f6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:29.132601 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb92694-733d-4b24-8b85-0391d0a4f6f5 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:29.133936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 650/1296] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:28:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:28:29.139133 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15e62b-6083-4265-aae0-31e37658ebc0 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:29.139550 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15e62b-6083-4265-aae0-31e37658ebc0 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:29.139688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15e62b-6083-4265-aae0-31e37658ebc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:29.156896 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2b15e62b-6083-4265-aae0-31e37658ebc0 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:28:29.158067 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2b15e62b-6083-4265-aae0-31e37658ebc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:28:29.238839 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 647/1297] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:28:29 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:28:29.264109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb92694-733d-4b24-8b85-0391d0a4f6f5 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:29.363868 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb92694-733d-4b24-8b85-0391d0a4f6f5 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:29.397475 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e07929-a431-43b7-a07c-a01cd71df85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:29.398275 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e07929-a431-43b7-a07c-a01cd71df85c None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:29.398555 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e07929-a431-43b7-a07c-a01cd71df85c None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:29.398826 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e07929-a431-43b7-a07c-a01cd71df85c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:29.399465 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e07929-a431-43b7-a07c-a01cd71df85c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:29.400876 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e07929-a431-43b7-a07c-a01cd71df85c None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:29.413000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-f3e07929-a431-43b7-a07c-a01cd71df85c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 10:28:29.413000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:29.413000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:28:29.413000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:28:29.413000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:28:29.413000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:28:29.413000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:28:29.413000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:28:29.413000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 10:28:29.413000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:28:29.413000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:28:29.413000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:28:29.413000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:29.413000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:29.413000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:29.413000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:29.413000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:29.414269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:29.414269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:29.414269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:29.414269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:29.414269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:29.414269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:29.414269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:28:29.414269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:28:29.414269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:28:29.414269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:28:29.414269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:29.414269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:29.414269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:29.414269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:29.414269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:29.414269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:29.415339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:29.415339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:29.415339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:29.415339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:29.415339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:29.415339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:29.415339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:29.415339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:29.415339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:29.415339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:29.415339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:28:29.415339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:28:29.415339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 10:28:29.415339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:28:29.415339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 651/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jun 6 10:28:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:29.420444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15e62b-6083-4265-aae0-31e37658ebc0 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:29.440190 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:29.520324 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15e62b-6083-4265-aae0-31e37658ebc0 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:29.554385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759215b1-165e-436f-a4ce-f643ff385387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:29.555034 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759215b1-165e-436f-a4ce-f643ff385387 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:29.555315 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759215b1-165e-436f-a4ce-f643ff385387 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:29.555590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759215b1-165e-436f-a4ce-f643ff385387 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:29.556314 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759215b1-165e-436f-a4ce-f643ff385387 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:29.556616 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759215b1-165e-436f-a4ce-f643ff385387 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:29.557966 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759215b1-165e-436f-a4ce-f643ff385387 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:29.562914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 648/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Jun 6 10:28:29 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:29.569034 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e07929-a431-43b7-a07c-a01cd71df85c None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:29.600687 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:29.668805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e07929-a431-43b7-a07c-a01cd71df85c None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:29.700938 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238fd48-0d9e-4682-862b-8cb147f76c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:29.701580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238fd48-0d9e-4682-862b-8cb147f76c4e None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:29.701858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238fd48-0d9e-4682-862b-8cb147f76c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:29.702128 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238fd48-0d9e-4682-862b-8cb147f76c4e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:29.702900 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238fd48-0d9e-4682-862b-8cb147f76c4e None admin] RBAC: Adding query filter params (interface=public, service_id=65510df4424d432eae10fd94f5fa27e5, region_id=RegionOne) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:29.703206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238fd48-0d9e-4682-862b-8cb147f76c4e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:29.704613 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238fd48-0d9e-4682-862b-8cb147f76c4e None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:29.710084 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 652/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jun 6 10:28:29 2022] GET /identity/v3/endpoints?service_id=65510df4424d432eae10fd94f5fa27e5&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:29.715755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759215b1-165e-436f-a4ce-f643ff385387 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:29.795428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:29.813160 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759215b1-165e-436f-a4ce-f643ff385387 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:29.846167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea26475-1370-44e3-8e54-c9413f8791ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:29.846819 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea26475-1370-44e3-8e54-c9413f8791ed None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:29.847105 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea26475-1370-44e3-8e54-c9413f8791ed None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:29.847385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea26475-1370-44e3-8e54-c9413f8791ed None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:29.848091 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea26475-1370-44e3-8e54-c9413f8791ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:29.849454 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea26475-1370-44e3-8e54-c9413f8791ed None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:29.856628 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 649/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jun 6 10:28:29 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:28:32.118283 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf512c-be70-42d0-b221-85b00e7a7500 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:32.118867 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf512c-be70-42d0-b221-85b00e7a7500 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:32.118867 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf512c-be70-42d0-b221-85b00e7a7500 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:32.120210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 653/1302] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:28:32.125411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e892f-bbf7-4186-807d-ebbf719d8bd3 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:32.125837 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e892f-bbf7-4186-807d-ebbf719d8bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:32.125961 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e892f-bbf7-4186-807d-ebbf719d8bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:32.143559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-6c8e892f-bbf7-4186-807d-ebbf719d8bd3 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:28:32.145017 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6c8e892f-bbf7-4186-807d-ebbf719d8bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:28:32.208474 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 650/1303] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:28:32 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 10:28:32.231142 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf512c-be70-42d0-b221-85b00e7a7500 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:32.322765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:32.329775 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf512c-be70-42d0-b221-85b00e7a7500 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:32.361598 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615fd90-785c-4139-91d1-2e5740896903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:32.362379 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615fd90-785c-4139-91d1-2e5740896903 None admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:32.362655 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615fd90-785c-4139-91d1-2e5740896903 None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:32.362925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615fd90-785c-4139-91d1-2e5740896903 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:32.363556 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f615fd90-785c-4139-91d1-2e5740896903 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:32.364924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f615fd90-785c-4139-91d1-2e5740896903 None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:32.376541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application [None req-f615fd90-785c-4139-91d1-2e5740896903 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 10:28:32.376541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:32.376541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 10:28:32.376541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 10:28:32.376541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 10:28:32.376541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 10:28:32.376541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 10:28:32.376541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:28:32.376541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 10:28:32.376541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:28:32.376541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 10:28:32.376541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:28:32.376541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 10:28:32.376541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 10:28:32.376541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 10:28:32.376541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 10:28:32.376541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 10:28:32.377661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 10:28:32.377661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 10:28:32.377661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 10:28:32.377661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 10:28:32.377661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 10:28:32.377661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 10:28:32.377661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 10:28:32.377661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 10:28:32.377661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 10:28:32.377661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 10:28:32.377661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 10:28:32.377661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 10:28:32.377661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 10:28:32.377661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 10:28:32.377661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 10:28:32.377661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 10:28:32.378589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 10:28:32.378589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application with Lock( Jun 06 10:28:32.378589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 10:28:32.378589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self._enter() Jun 06 10:28:32.378589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 10:28:32.378589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 10:28:32.378589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 10:28:32.378589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application return self.creator() Jun 06 10:28:32.378589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 10:28:32.378589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application created_value = creator( Jun 06 10:28:32.378589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 10:28:32.378589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 10:28:32.378589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 10:28:32.378589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: ERROR keystone.server.flask.application Jun 06 10:28:32.380017 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 654/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jun 6 10:28:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 10:28:32.384564 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e892f-bbf7-4186-807d-ebbf719d8bd3 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:32.456262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:32.479103 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e892f-bbf7-4186-807d-ebbf719d8bd3 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:32.512714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1542f81-ca09-470c-bc3f-488d9dda5c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:32.513363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1542f81-ca09-470c-bc3f-488d9dda5c85 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:32.513644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1542f81-ca09-470c-bc3f-488d9dda5c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:32.513918 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1542f81-ca09-470c-bc3f-488d9dda5c85 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:32.514615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1542f81-ca09-470c-bc3f-488d9dda5c85 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:28:32.514917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1542f81-ca09-470c-bc3f-488d9dda5c85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:32.516305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1542f81-ca09-470c-bc3f-488d9dda5c85 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:32.521548 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 651/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Jun 6 10:28:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:28:32.527514 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615fd90-785c-4139-91d1-2e5740896903 None admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:28:32.624461 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615fd90-785c-4139-91d1-2e5740896903 None admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:28:32.657069 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fa9c6-f761-4c21-9309-77afadd9606d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:28:32.657645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fa9c6-f761-4c21-9309-77afadd9606d None admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:28:32.657928 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fa9c6-f761-4c21-9309-77afadd9606d None admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:28:32.658214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fa9c6-f761-4c21-9309-77afadd9606d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:28:32.658939 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6fa9c6-f761-4c21-9309-77afadd9606d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:28:32.660327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6fa9c6-f761-4c21-9309-77afadd9606d None admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:28:32.677785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 655/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jun 6 10:28:32 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:28:42.518279 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=30 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:28:42.673600 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:29:36.713361 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bb0a5-e260-49f8-bf24-44847b5ebe14 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:36.713834 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bb0a5-e260-49f8-bf24-44847b5ebe14 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:36.714051 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bb0a5-e260-49f8-bf24-44847b5ebe14 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:36.718999 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 652/1307] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:29:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:29:36.720490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4b452-acb5-4ff8-bca3-9c97596cc18f None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:36.720924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4b452-acb5-4ff8-bca3-9c97596cc18f None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:36.721077 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4b452-acb5-4ff8-bca3-9c97596cc18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:36.883659 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-aba4b452-acb5-4ff8-bca3-9c97596cc18f None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:29:36.884953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-aba4b452-acb5-4ff8-bca3-9c97596cc18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:37.019146 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 656/1308] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:29:37.071868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26d5c1-ec96-4d86-9976-95f766fca29a None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:37.072149 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26d5c1-ec96-4d86-9976-95f766fca29a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:37.072419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26d5c1-ec96-4d86-9976-95f766fca29a None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:37.073944 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 653/1309] 10.222.0.42 () {54 vars in 964 bytes} [Mon Jun 6 10:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:29:37.080403 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cb996-c134-4b2a-850c-0661768cf61d None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:37.080732 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cb996-c134-4b2a-850c-0661768cf61d None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:37.080968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cb996-c134-4b2a-850c-0661768cf61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:37.166389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-108cb996-c134-4b2a-850c-0661768cf61d None None] MFA Rules not processed for user `881069f9b40a485dac2dda684dd6cd3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:29:37.167623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-108cb996-c134-4b2a-850c-0661768cf61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:37.190532 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:29:37.251659 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 657/1310] 10.222.0.42 () {58 vars in 1061 bytes} [Mon Jun 6 10:29:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:29:37.260645 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26d5c1-ec96-4d86-9976-95f766fca29a None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:29:37.262408 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ad26d5c1-ec96-4d86-9976-95f766fca29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:37.380705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26d5c1-ec96-4d86-9976-95f766fca29a None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:29:37.461811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c1b86-5099-4a99-9fa0-6393b2604716 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:29:37.462464 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c1b86-5099-4a99-9fa0-6393b2604716 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:37.462745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c1b86-5099-4a99-9fa0-6393b2604716 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:37.463021 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c1b86-5099-4a99-9fa0-6393b2604716 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:37.464707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-522c1b86-5099-4a99-9fa0-6393b2604716 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:37.521216 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:29:37.578888 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c1b86-5099-4a99-9fa0-6393b2604716 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:29:37.580943 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c1b86-5099-4a99-9fa0-6393b2604716 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:29:37.600566 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 654/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:29:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:29:40.166494 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7743a44-1f16-4ec9-9a07-405816204a39 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:40.167053 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7743a44-1f16-4ec9-9a07-405816204a39 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:40.167053 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7743a44-1f16-4ec9-9a07-405816204a39 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:40.168448 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 658/1312] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:29:40.174532 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a9e64-6d89-4fe8-b6c5-7cacd8a6108f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:40.175168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a9e64-6d89-4fe8-b6c5-7cacd8a6108f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:40.175168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a9e64-6d89-4fe8-b6c5-7cacd8a6108f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:40.252602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-d66a9e64-6d89-4fe8-b6c5-7cacd8a6108f None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:29:40.253900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d66a9e64-6d89-4fe8-b6c5-7cacd8a6108f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:40.363935 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 655/1313] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:29:40.401807 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7743a44-1f16-4ec9-9a07-405816204a39 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:29:40.403364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a7743a44-1f16-4ec9-9a07-405816204a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:40.512417 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7743a44-1f16-4ec9-9a07-405816204a39 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:29:40.587649 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:29:40.590552 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1c481-52c3-4a61-8345-0a53503f766e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:29:40.591426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1c481-52c3-4a61-8345-0a53503f766e service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:40.592110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1c481-52c3-4a61-8345-0a53503f766e service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:40.592715 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1c481-52c3-4a61-8345-0a53503f766e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:40.594687 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-36f1c481-52c3-4a61-8345-0a53503f766e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:40.710928 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1c481-52c3-4a61-8345-0a53503f766e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:29:40.712567 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1c481-52c3-4a61-8345-0a53503f766e service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:29:40.732368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 659/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:29:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:29:43.374195 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244580cd-9eec-4bef-bea3-78acc7ec056c None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:43.374696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244580cd-9eec-4bef-bea3-78acc7ec056c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:43.374696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244580cd-9eec-4bef-bea3-78acc7ec056c None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:43.376064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 656/1315] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:29:43.384684 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaae2be-a7fb-4790-9972-21ddae3250dd None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:43.385093 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaae2be-a7fb-4790-9972-21ddae3250dd None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:43.385093 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaae2be-a7fb-4790-9972-21ddae3250dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:43.451423 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-0aaae2be-a7fb-4790-9972-21ddae3250dd None None] MFA Rules not processed for user `709a0882f46a4bcaa3f47007f20cfaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:29:43.452719 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-0aaae2be-a7fb-4790-9972-21ddae3250dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:43.503354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:29:43.547677 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 660/1316] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:29:43 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:29:43.588134 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244580cd-9eec-4bef-bea3-78acc7ec056c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:29:43.589667 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-244580cd-9eec-4bef-bea3-78acc7ec056c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:43.599602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:29:43.696126 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244580cd-9eec-4bef-bea3-78acc7ec056c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:29:43.768521 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c99043-e690-4927-a9cb-006cf5c3469e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:29:43.769161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c99043-e690-4927-a9cb-006cf5c3469e service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:43.769442 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c99043-e690-4927-a9cb-006cf5c3469e service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:43.769716 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c99043-e690-4927-a9cb-006cf5c3469e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:43.771338 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b7c99043-e690-4927-a9cb-006cf5c3469e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:43.832410 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:29:43.878208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c99043-e690-4927-a9cb-006cf5c3469e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:29:43.879631 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c99043-e690-4927-a9cb-006cf5c3469e service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:29:43.906879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 657/1317] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:29:43 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:29:46.725746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a53a4-3efc-40e5-93eb-35dad67cb93f None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:46.726272 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a53a4-3efc-40e5-93eb-35dad67cb93f None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:46.726368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a53a4-3efc-40e5-93eb-35dad67cb93f None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:46.727677 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 661/1318] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:29:46.732729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c59b06-a6e0-4872-a7d9-e3788dbbbb5c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:46.733124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c59b06-a6e0-4872-a7d9-e3788dbbbb5c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:46.733279 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c59b06-a6e0-4872-a7d9-e3788dbbbb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:46.799613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-77c59b06-a6e0-4872-a7d9-e3788dbbbb5c None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:29:46.800846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-77c59b06-a6e0-4872-a7d9-e3788dbbbb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:46.895473 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 658/1319] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:29:46 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:29:46.934978 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a53a4-3efc-40e5-93eb-35dad67cb93f None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:29:46.936554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-910a53a4-3efc-40e5-93eb-35dad67cb93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:47.038761 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a53a4-3efc-40e5-93eb-35dad67cb93f None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:29:47.115266 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1e4a8-c031-4d41-9b79-5fed7d6b7170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:29:47.115954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1e4a8-c031-4d41-9b79-5fed7d6b7170 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:47.116242 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1e4a8-c031-4d41-9b79-5fed7d6b7170 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:47.116520 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1e4a8-c031-4d41-9b79-5fed7d6b7170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:47.118175 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-2eb1e4a8-c031-4d41-9b79-5fed7d6b7170 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:47.125811 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:29:47.226891 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1e4a8-c031-4d41-9b79-5fed7d6b7170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:29:47.228440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1e4a8-c031-4d41-9b79-5fed7d6b7170 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:29:47.248001 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 662/1320] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:29:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:29:50.152765 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85176490-0540-4d79-98c5-d0e15b07980f None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:50.153348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85176490-0540-4d79-98c5-d0e15b07980f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:50.153348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85176490-0540-4d79-98c5-d0e15b07980f None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:50.154655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 659/1321] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:29:50.160396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830acbb6-f3a4-4faf-a440-49ca2420100c None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:50.160812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830acbb6-f3a4-4faf-a440-49ca2420100c None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:50.160941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830acbb6-f3a4-4faf-a440-49ca2420100c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:50.227534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-830acbb6-f3a4-4faf-a440-49ca2420100c None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:29:50.228783 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-830acbb6-f3a4-4faf-a440-49ca2420100c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:50.285963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:29:50.325142 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 663/1322] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:29:50.361850 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85176490-0540-4d79-98c5-d0e15b07980f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:29:50.363374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-85176490-0540-4d79-98c5-d0e15b07980f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:50.385237 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:29:50.470999 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85176490-0540-4d79-98c5-d0e15b07980f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:29:50.543769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aaa975-9c8a-4aca-a089-078d11b582f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:29:50.544445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aaa975-9c8a-4aca-a089-078d11b582f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:50.544727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aaa975-9c8a-4aca-a089-078d11b582f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:50.544998 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aaa975-9c8a-4aca-a089-078d11b582f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:50.546636 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-78aaa975-9c8a-4aca-a089-078d11b582f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:50.617193 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:29:50.658641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aaa975-9c8a-4aca-a089-078d11b582f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:29:50.660095 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aaa975-9c8a-4aca-a089-078d11b582f8 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:29:50.677989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 660/1323] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:29:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:29:53.021069 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a9ee2-5779-4f25-bebd-ebe1ed30cc85 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:53.021604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a9ee2-5779-4f25-bebd-ebe1ed30cc85 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:53.021604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a9ee2-5779-4f25-bebd-ebe1ed30cc85 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:53.022992 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 664/1324] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:29:53.028587 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4674c-e74e-4123-b155-ac377eb4dbbb None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:53.029023 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4674c-e74e-4123-b155-ac377eb4dbbb None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:53.029106 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4674c-e74e-4123-b155-ac377eb4dbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:53.092533 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-0ea4674c-e74e-4123-b155-ac377eb4dbbb None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:29:53.093663 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0ea4674c-e74e-4123-b155-ac377eb4dbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:53.193401 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 661/1325] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:29:53.231755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a9ee2-5779-4f25-bebd-ebe1ed30cc85 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:29:53.233312 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-656a9ee2-5779-4f25-bebd-ebe1ed30cc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:53.340494 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a9ee2-5779-4f25-bebd-ebe1ed30cc85 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:29:53.418314 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7823b7-95e5-4b3e-956d-3e3e9bf0031d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:29:53.418958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7823b7-95e5-4b3e-956d-3e3e9bf0031d service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:53.419240 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7823b7-95e5-4b3e-956d-3e3e9bf0031d service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:53.419519 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7823b7-95e5-4b3e-956d-3e3e9bf0031d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:53.421155 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-2d7823b7-95e5-4b3e-956d-3e3e9bf0031d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:53.438729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:29:53.534002 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7823b7-95e5-4b3e-956d-3e3e9bf0031d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:29:53.535428 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7823b7-95e5-4b3e-956d-3e3e9bf0031d service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:29:53.555985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 665/1326] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:29:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:29:55.898380 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722ade4-1e50-4cc9-9992-1eabaacea68f None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:55.898893 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722ade4-1e50-4cc9-9992-1eabaacea68f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:55.898893 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722ade4-1e50-4cc9-9992-1eabaacea68f None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:55.900237 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 662/1327] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:29:55.905557 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a8b90-cd9d-422f-bf1a-9d8f1578c7f3 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:55.906000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a8b90-cd9d-422f-bf1a-9d8f1578c7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:55.906118 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a8b90-cd9d-422f-bf1a-9d8f1578c7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:55.986469 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-5a1a8b90-cd9d-422f-bf1a-9d8f1578c7f3 None None] MFA Rules not processed for user `709a0882f46a4bcaa3f47007f20cfaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:29:55.987704 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-5a1a8b90-cd9d-422f-bf1a-9d8f1578c7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:56.062440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:29:56.103776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 666/1328] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:29:55 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:29:56.163763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff66bbb-f5ea-480a-a8c6-a37f5acbf6c4 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:56.164100 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff66bbb-f5ea-480a-a8c6-a37f5acbf6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:56.164466 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff66bbb-f5ea-480a-a8c6-a37f5acbf6c4 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:56.165915 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 663/1329] 10.222.0.42 () {54 vars in 964 bytes} [Mon Jun 6 10:29:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:29:56.172401 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72ad12-1037-470f-af9b-bc8bb262a12e None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:56.172690 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72ad12-1037-470f-af9b-bc8bb262a12e None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:56.172966 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72ad12-1037-470f-af9b-bc8bb262a12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:56.255667 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-5d72ad12-1037-470f-af9b-bc8bb262a12e None None] MFA Rules not processed for user `881069f9b40a485dac2dda684dd6cd3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:29:56.257153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-5d72ad12-1037-470f-af9b-bc8bb262a12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:56.349113 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 667/1330] 10.222.0.42 () {58 vars in 1061 bytes} [Mon Jun 6 10:29:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:29:56.358516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff66bbb-f5ea-480a-a8c6-a37f5acbf6c4 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:29:56.360755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-eff66bbb-f5ea-480a-a8c6-a37f5acbf6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:56.401172 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:29:56.491431 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff66bbb-f5ea-480a-a8c6-a37f5acbf6c4 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:29:56.584805 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89579c04-fa88-4b0b-9533-f52b5cf74f73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:29:56.585457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89579c04-fa88-4b0b-9533-f52b5cf74f73 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:56.585739 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89579c04-fa88-4b0b-9533-f52b5cf74f73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:56.586010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89579c04-fa88-4b0b-9533-f52b5cf74f73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:56.587649 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-89579c04-fa88-4b0b-9533-f52b5cf74f73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:56.665052 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:29:56.698243 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89579c04-fa88-4b0b-9533-f52b5cf74f73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:29:56.699671 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89579c04-fa88-4b0b-9533-f52b5cf74f73 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:29:56.717558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 664/1331] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:29:56 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:29:59.609758 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac21573-ba16-4b74-955d-84848bc99964 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:59.610324 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac21573-ba16-4b74-955d-84848bc99964 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:59.610324 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac21573-ba16-4b74-955d-84848bc99964 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:59.612208 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 668/1332] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:29:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:29:59.617438 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab94cda-bdc2-401a-b330-7a98f45d51d0 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:29:59.617878 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab94cda-bdc2-401a-b330-7a98f45d51d0 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:29:59.617986 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab94cda-bdc2-401a-b330-7a98f45d51d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:29:59.691293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3ab94cda-bdc2-401a-b330-7a98f45d51d0 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:29:59.692536 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3ab94cda-bdc2-401a-b330-7a98f45d51d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:59.793507 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 665/1333] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:29:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:29:59.830684 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac21573-ba16-4b74-955d-84848bc99964 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:29:59.832339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-4ac21573-ba16-4b74-955d-84848bc99964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:29:59.904455 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:29:59.952319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac21573-ba16-4b74-955d-84848bc99964 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:30:00.029957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4e50c-5115-4d63-ae4c-6326ede60c4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:30:00.030600 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4e50c-5115-4d63-ae4c-6326ede60c4f service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:00.030881 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4e50c-5115-4d63-ae4c-6326ede60c4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:00.031388 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4e50c-5115-4d63-ae4c-6326ede60c4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:00.033112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-1bd4e50c-5115-4d63-ae4c-6326ede60c4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:00.145047 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4e50c-5115-4d63-ae4c-6326ede60c4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:30:00.146375 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4e50c-5115-4d63-ae4c-6326ede60c4f service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:30:00.156988 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:30:00.166301 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 669/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:29:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:30:02.472167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e85021-e74a-4825-b677-62f2a59224f2 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:02.472717 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e85021-e74a-4825-b677-62f2a59224f2 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:02.472717 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e85021-e74a-4825-b677-62f2a59224f2 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:02.474058 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 666/1335] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:30:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:02.479274 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d78ed5-9353-4a71-a0ba-dbae53887fa4 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:02.479752 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d78ed5-9353-4a71-a0ba-dbae53887fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:02.479864 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d78ed5-9353-4a71-a0ba-dbae53887fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:02.543530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-42d78ed5-9353-4a71-a0ba-dbae53887fa4 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:02.544982 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-42d78ed5-9353-4a71-a0ba-dbae53887fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:02.642838 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 670/1336] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:30:02.683216 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e85021-e74a-4825-b677-62f2a59224f2 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:30:02.684813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-81e85021-e74a-4825-b677-62f2a59224f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:02.724282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:30:02.789160 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e85021-e74a-4825-b677-62f2a59224f2 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:30:02.865373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a94ca2-a3e9-4816-8555-7ed16381fa64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:30:02.866025 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a94ca2-a3e9-4816-8555-7ed16381fa64 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:02.866306 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a94ca2-a3e9-4816-8555-7ed16381fa64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:02.866576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a94ca2-a3e9-4816-8555-7ed16381fa64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:02.868264 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-22a94ca2-a3e9-4816-8555-7ed16381fa64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:02.953430 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:30:02.980995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a94ca2-a3e9-4816-8555-7ed16381fa64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:30:02.982436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a94ca2-a3e9-4816-8555-7ed16381fa64 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:30:03.000502 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 667/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:30:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:30:05.752123 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad910910-4109-4c2b-919d-3a156f3ba2f1 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:05.752742 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad910910-4109-4c2b-919d-3a156f3ba2f1 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:05.752742 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad910910-4109-4c2b-919d-3a156f3ba2f1 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:05.754039 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 671/1338] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:05.759302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891134fb-b874-44f9-98c2-54f7e72bf7e6 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:05.759719 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891134fb-b874-44f9-98c2-54f7e72bf7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:05.759891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891134fb-b874-44f9-98c2-54f7e72bf7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:05.823998 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-891134fb-b874-44f9-98c2-54f7e72bf7e6 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:05.825198 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-891134fb-b874-44f9-98c2-54f7e72bf7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:05.920097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 668/1339] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:30:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:30:05.957663 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad910910-4109-4c2b-919d-3a156f3ba2f1 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:30:05.959228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ad910910-4109-4c2b-919d-3a156f3ba2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:06.036022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:30:06.063509 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad910910-4109-4c2b-919d-3a156f3ba2f1 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:30:06.135236 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e30c7f-0064-472e-8a23-01091e01fcdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:30:06.135914 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e30c7f-0064-472e-8a23-01091e01fcdf service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:06.136193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e30c7f-0064-472e-8a23-01091e01fcdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:06.136463 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e30c7f-0064-472e-8a23-01091e01fcdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:06.138074 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a6e30c7f-0064-472e-8a23-01091e01fcdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:06.245342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e30c7f-0064-472e-8a23-01091e01fcdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:30:06.246807 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e30c7f-0064-472e-8a23-01091e01fcdf service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:30:06.265958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 672/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:30:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:30:10.629739 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ef58c-ef63-42fb-9251-9e0a951c2ffe None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:10.630298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ef58c-ef63-42fb-9251-9e0a951c2ffe None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:10.630298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ef58c-ef63-42fb-9251-9e0a951c2ffe None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:10.631701 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 669/1341] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:10.637095 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b02f86-dfe7-4383-81b8-58d5d14077ce None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:10.637501 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b02f86-dfe7-4383-81b8-58d5d14077ce None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:10.637664 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b02f86-dfe7-4383-81b8-58d5d14077ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:10.644948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:30:10.710981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-e2b02f86-dfe7-4383-81b8-58d5d14077ce None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:10.712151 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-e2b02f86-dfe7-4383-81b8-58d5d14077ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:10.818219 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 673/1342] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:30:10.856291 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ef58c-ef63-42fb-9251-9e0a951c2ffe None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:30:10.858449 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6a0ef58c-ef63-42fb-9251-9e0a951c2ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:10.913662 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:30:10.961227 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ef58c-ef63-42fb-9251-9e0a951c2ffe None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:30:10.964679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2ea38-8123-49bc-a12c-d1bf6f91104d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:30:10.965322 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2ea38-8123-49bc-a12c-d1bf6f91104d service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:10.965601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2ea38-8123-49bc-a12c-d1bf6f91104d service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:10.965870 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2ea38-8123-49bc-a12c-d1bf6f91104d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:10.975361 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2ea38-8123-49bc-a12c-d1bf6f91104d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:30:10.976827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2ea38-8123-49bc-a12c-d1bf6f91104d service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:30:10.987725 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 670/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:30:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:30:13.912537 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5f2fd-66c6-445f-85d2-d36c2ee87eb5 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:13.913062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5f2fd-66c6-445f-85d2-d36c2ee87eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:13.913062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5f2fd-66c6-445f-85d2-d36c2ee87eb5 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:13.914401 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 674/1344] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:13.919672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9a9fd-5f98-4093-bfd7-d90966806142 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:13.920127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9a9fd-5f98-4093-bfd7-d90966806142 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:13.920230 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9a9fd-5f98-4093-bfd7-d90966806142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:13.947048 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-76c9a9fd-5f98-4093-bfd7-d90966806142 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:13.948250 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-76c9a9fd-5f98-4093-bfd7-d90966806142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:14.027134 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 671/1345] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:30:13 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:30:14.070865 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5f2fd-66c6-445f-85d2-d36c2ee87eb5 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:30:14.082481 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5f2fd-66c6-445f-85d2-d36c2ee87eb5 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:30:14.086116 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f83b3-0989-4d13-9862-5d2d74e548e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:30:14.086825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f83b3-0989-4d13-9862-5d2d74e548e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:14.087179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f83b3-0989-4d13-9862-5d2d74e548e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:14.087526 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f83b3-0989-4d13-9862-5d2d74e548e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:14.097099 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5f83b3-0989-4d13-9862-5d2d74e548e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:30:14.098594 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5f83b3-0989-4d13-9862-5d2d74e548e9 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:30:14.108083 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 675/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:30:14 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:30:18.301185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875fa49-1778-489d-9f44-de9b18543901 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:18.301683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875fa49-1778-489d-9f44-de9b18543901 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:18.301786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875fa49-1778-489d-9f44-de9b18543901 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:18.303326 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 672/1347] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:18.308603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee849f-d5b3-4590-a3f9-2ac4d7f013aa None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:18.309078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee849f-d5b3-4590-a3f9-2ac4d7f013aa None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:18.309177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee849f-d5b3-4590-a3f9-2ac4d7f013aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:18.327354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-b3ee849f-d5b3-4590-a3f9-2ac4d7f013aa None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:18.328596 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-b3ee849f-d5b3-4590-a3f9-2ac4d7f013aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:18.337680 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 676/1348] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:30:18.375503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875fa49-1778-489d-9f44-de9b18543901 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:30:18.387302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875fa49-1778-489d-9f44-de9b18543901 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:30:18.390864 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa96a1-ccc2-476f-8d27-20a28672427a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:30:18.391564 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa96a1-ccc2-476f-8d27-20a28672427a service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:18.392462 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa96a1-ccc2-476f-8d27-20a28672427a service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:18.393689 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa96a1-ccc2-476f-8d27-20a28672427a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:18.403089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa96a1-ccc2-476f-8d27-20a28672427a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:30:18.404618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa96a1-ccc2-476f-8d27-20a28672427a service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:30:18.415281 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 673/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:30:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:30:24.119731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba507a-06e1-4570-891e-4a077de6841d None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:24.120936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba507a-06e1-4570-891e-4a077de6841d None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:24.121422 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba507a-06e1-4570-891e-4a077de6841d None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:24.123355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 677/1350] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jun 6 10:30:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:24.129223 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114513ac-c9d6-416c-ac2f-bbac1ff0d8bf None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:24.130153 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114513ac-c9d6-416c-ac2f-bbac1ff0d8bf None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:24.130703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114513ac-c9d6-416c-ac2f-bbac1ff0d8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:24.409582 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-114513ac-c9d6-416c-ac2f-bbac1ff0d8bf None None] MFA Rules not processed for user `361be69f1c0a41c18afd206bccf56447`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:24.410774 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-114513ac-c9d6-416c-ac2f-bbac1ff0d8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:24.474799 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:30:24.553094 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 674/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jun 6 10:30:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:30:28.323171 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=72 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:30:29.070979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41bc3f-ce70-4e0c-bdb0-553595634117 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:29.071508 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41bc3f-ce70-4e0c-bdb0-553595634117 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:29.072093 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41bc3f-ce70-4e0c-bdb0-553595634117 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:29.073906 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 678/1352] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jun 6 10:30:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:29.080197 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1659b-1ec4-432a-b4a5-ad9f97efc09b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:29.080716 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1659b-1ec4-432a-b4a5-ad9f97efc09b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:29.081423 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1659b-1ec4-432a-b4a5-ad9f97efc09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:29.102649 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b9f1659b-1ec4-432a-b4a5-ad9f97efc09b None None] MFA Rules not processed for user `361be69f1c0a41c18afd206bccf56447`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:29.104121 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b9f1659b-1ec4-432a-b4a5-ad9f97efc09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:29.115127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 675/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jun 6 10:30:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:30:30.812584 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291beb4e-4ffe-43f4-9874-f90b93d84b28 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:30.813065 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291beb4e-4ffe-43f4-9874-f90b93d84b28 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:30.813276 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291beb4e-4ffe-43f4-9874-f90b93d84b28 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:30.814994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 679/1354] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jun 6 10:30:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:30.820225 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bb391-e6ce-4bd7-a8e8-76e09595907e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:30.820667 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bb391-e6ce-4bd7-a8e8-76e09595907e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:30.820833 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bb391-e6ce-4bd7-a8e8-76e09595907e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:30.839847 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-963bb391-e6ce-4bd7-a8e8-76e09595907e None None] MFA Rules not processed for user `361be69f1c0a41c18afd206bccf56447`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:30.841281 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-963bb391-e6ce-4bd7-a8e8-76e09595907e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:30.850774 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 676/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jun 6 10:30:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:30:34.518268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d5b92-4cb0-4cd7-ae29-cadc4c2a4a30 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:34.519484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d5b92-4cb0-4cd7-ae29-cadc4c2a4a30 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:34.520010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d5b92-4cb0-4cd7-ae29-cadc4c2a4a30 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:34.521570 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 680/1356] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:30:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:34.528961 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e46f7d-f17e-4e7d-8551-132e2fd0d233 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:34.529495 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e46f7d-f17e-4e7d-8551-132e2fd0d233 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:34.529738 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e46f7d-f17e-4e7d-8551-132e2fd0d233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:34.558533 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-81e46f7d-f17e-4e7d-8551-132e2fd0d233 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:34.561040 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-81e46f7d-f17e-4e7d-8551-132e2fd0d233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:34.572286 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 677/1357] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:30:34.638238 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32bbeb-344a-49c0-a555-bd83cb627160 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:34.638816 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32bbeb-344a-49c0-a555-bd83cb627160 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:34.639338 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32bbeb-344a-49c0-a555-bd83cb627160 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:34.641181 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 681/1358] 10.222.0.42 () {54 vars in 961 bytes} [Mon Jun 6 10:30:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:34.657196 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ad1ef-d4f6-412c-86ca-054ecd08fe47 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:34.657855 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ad1ef-d4f6-412c-86ca-054ecd08fe47 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:34.658210 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ad1ef-d4f6-412c-86ca-054ecd08fe47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:34.734427 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-f35ad1ef-d4f6-412c-86ca-054ecd08fe47 None None] MFA Rules not processed for user `ed8a40e4f1034a8e99d0eb505e448db9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:34.735694 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f35ad1ef-d4f6-412c-86ca-054ecd08fe47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:34.868124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 678/1359] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Jun 6 10:30:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:30:34.873978 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32bbeb-344a-49c0-a555-bd83cb627160 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:30:34.894003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32bbeb-344a-49c0-a555-bd83cb627160 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:30:34.900395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229dc38b-b26c-4bdc-a588-469865c00457 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:30:34.901316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dc38b-b26c-4bdc-a588-469865c00457 service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:34.901754 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dc38b-b26c-4bdc-a588-469865c00457 service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:34.902199 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dc38b-b26c-4bdc-a588-469865c00457 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:34.912670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229dc38b-b26c-4bdc-a588-469865c00457 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:30:34.914673 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229dc38b-b26c-4bdc-a588-469865c00457 service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:30:34.928488 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 682/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:30:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:30:37.504596 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6e865-6235-4fb1-809e-34c26e29e801 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:37.505148 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6e865-6235-4fb1-809e-34c26e29e801 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:37.505148 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6e865-6235-4fb1-809e-34c26e29e801 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:37.506673 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 679/1361] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:37.512106 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c3a43-3954-4d06-a8cb-c690c39972da None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:37.512648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c3a43-3954-4d06-a8cb-c690c39972da None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:37.512648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c3a43-3954-4d06-a8cb-c690c39972da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:37.534033 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-009c3a43-3954-4d06-a8cb-c690c39972da None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:37.535379 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-009c3a43-3954-4d06-a8cb-c690c39972da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:37.544029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 683/1362] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:30:37.590501 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6e865-6235-4fb1-809e-34c26e29e801 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:30:37.606208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6e865-6235-4fb1-809e-34c26e29e801 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:30:37.612667 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0f2d2-60e7-41d9-b38c-90ee843faef3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:30:37.613389 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0f2d2-60e7-41d9-b38c-90ee843faef3 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:37.613737 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0f2d2-60e7-41d9-b38c-90ee843faef3 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:37.614080 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0f2d2-60e7-41d9-b38c-90ee843faef3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:37.623528 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0f2d2-60e7-41d9-b38c-90ee843faef3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:30:37.625230 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0f2d2-60e7-41d9-b38c-90ee843faef3 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:30:37.635901 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 680/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:30:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:30:40.298219 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d9061-2a49-4bf7-849d-2bf841359908 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:40.299403 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d9061-2a49-4bf7-849d-2bf841359908 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:40.299403 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d9061-2a49-4bf7-849d-2bf841359908 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:40.300780 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 684/1364] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:30:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:40.306286 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a28d3-73b2-4bee-9522-084619d2c37c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:40.306713 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a28d3-73b2-4bee-9522-084619d2c37c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:40.306824 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a28d3-73b2-4bee-9522-084619d2c37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:40.326821 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-823a28d3-73b2-4bee-9522-084619d2c37c None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:40.327980 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-823a28d3-73b2-4bee-9522-084619d2c37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:40.336247 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 681/1365] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:30:40.383335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba676c-2289-4bd8-b04b-c5f7c8db4d27 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:40.383626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba676c-2289-4bd8-b04b-c5f7c8db4d27 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:40.383941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba676c-2289-4bd8-b04b-c5f7c8db4d27 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:40.386214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 685/1366] 10.222.0.42 () {54 vars in 961 bytes} [Mon Jun 6 10:30:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:40.391658 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf687d-0bfc-469f-b024-559a6120e895 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:40.391995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf687d-0bfc-469f-b024-559a6120e895 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:40.392438 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf687d-0bfc-469f-b024-559a6120e895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:40.411005 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9eaf687d-0bfc-469f-b024-559a6120e895 None None] MFA Rules not processed for user `ed8a40e4f1034a8e99d0eb505e448db9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:40.412230 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9eaf687d-0bfc-469f-b024-559a6120e895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:40.421370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 682/1367] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Jun 6 10:30:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:30:40.429739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba676c-2289-4bd8-b04b-c5f7c8db4d27 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:30:40.444346 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba676c-2289-4bd8-b04b-c5f7c8db4d27 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:30:40.453132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad0e16-f7c3-4361-ba95-6cfaf53b85e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:30:40.453967 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad0e16-f7c3-4361-ba95-6cfaf53b85e1 service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:40.454173 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad0e16-f7c3-4361-ba95-6cfaf53b85e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:40.454501 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad0e16-f7c3-4361-ba95-6cfaf53b85e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:40.465892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad0e16-f7c3-4361-ba95-6cfaf53b85e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:30:40.467125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad0e16-f7c3-4361-ba95-6cfaf53b85e1 service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:30:40.478002 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 686/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:30:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:30:42.847901 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45417764-e70a-4e6d-888b-ecc3e04b3e5e None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:42.848454 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45417764-e70a-4e6d-888b-ecc3e04b3e5e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:42.848454 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45417764-e70a-4e6d-888b-ecc3e04b3e5e None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:42.849820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 683/1369] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:42.855195 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7e26f-7f06-4c94-b73a-f630cf1b8415 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:42.855607 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7e26f-7f06-4c94-b73a-f630cf1b8415 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:42.855748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7e26f-7f06-4c94-b73a-f630cf1b8415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:42.874604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-27d7e26f-7f06-4c94-b73a-f630cf1b8415 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:42.875838 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-27d7e26f-7f06-4c94-b73a-f630cf1b8415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:42.884746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 687/1370] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:30:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:30:42.930031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45417764-e70a-4e6d-888b-ecc3e04b3e5e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:30:42.946461 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45417764-e70a-4e6d-888b-ecc3e04b3e5e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:30:42.952467 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29bc59-31a0-4fe1-99a7-48f6fae88115 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:30:42.953256 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29bc59-31a0-4fe1-99a7-48f6fae88115 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:42.953616 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29bc59-31a0-4fe1-99a7-48f6fae88115 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:42.953982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29bc59-31a0-4fe1-99a7-48f6fae88115 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:42.964735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc29bc59-31a0-4fe1-99a7-48f6fae88115 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:30:42.966395 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc29bc59-31a0-4fe1-99a7-48f6fae88115 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:30:42.977871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 684/1371] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:30:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:30:43.290663 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ce69e-385d-4759-bb39-407f0bc1b1d0 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:43.290947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ce69e-385d-4759-bb39-407f0bc1b1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:43.291223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ce69e-385d-4759-bb39-407f0bc1b1d0 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:43.293027 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 688/1372] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:43.298997 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fc645-6a14-4001-9d91-25674614d535 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:43.299294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fc645-6a14-4001-9d91-25674614d535 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:43.299573 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fc645-6a14-4001-9d91-25674614d535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:43.319161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-d95fc645-6a14-4001-9d91-25674614d535 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:43.320696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d95fc645-6a14-4001-9d91-25674614d535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:43.328871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 685/1373] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:30:43.602498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc761a5-15af-4589-8aad-f916223abba7 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:43.602784 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc761a5-15af-4589-8aad-f916223abba7 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:43.603060 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc761a5-15af-4589-8aad-f916223abba7 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:43.604682 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 689/1374] 10.222.0.42 () {54 vars in 963 bytes} [Mon Jun 6 10:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:43.609694 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c71653-8d28-40a2-8543-4e825dd39824 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:43.610248 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c71653-8d28-40a2-8543-4e825dd39824 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:43.610619 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c71653-8d28-40a2-8543-4e825dd39824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:43.618553 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:30:43.664087 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-18c71653-8d28-40a2-8543-4e825dd39824 None None] MFA Rules not processed for user `2b8836193f1a47e1a62ea5167e01b26e`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:43.665280 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-18c71653-8d28-40a2-8543-4e825dd39824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:43.735470 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 686/1375] 10.222.0.42 () {58 vars in 1060 bytes} [Mon Jun 6 10:30:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:30:43.743815 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc761a5-15af-4589-8aad-f916223abba7 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:30:43.755496 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc761a5-15af-4589-8aad-f916223abba7 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:30:43.759006 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922faa82-a925-4b3d-9a9a-447394ee3595 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:30:43.759655 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922faa82-a925-4b3d-9a9a-447394ee3595 service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:43.759985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922faa82-a925-4b3d-9a9a-447394ee3595 service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:43.760261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922faa82-a925-4b3d-9a9a-447394ee3595 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:43.771443 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922faa82-a925-4b3d-9a9a-447394ee3595 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:30:43.772929 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922faa82-a925-4b3d-9a9a-447394ee3595 service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:30:43.782923 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 690/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:30:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:30:45.319431 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafc622-7a14-45d5-baf9-3f7f7d4f065e None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:45.320025 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafc622-7a14-45d5-baf9-3f7f7d4f065e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:45.320025 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafc622-7a14-45d5-baf9-3f7f7d4f065e None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:45.321437 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 687/1377] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:45.326771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d0386-e4cd-4bff-865e-fe8d29b1cbdb None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:45.327198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d0386-e4cd-4bff-865e-fe8d29b1cbdb None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:45.327329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d0386-e4cd-4bff-865e-fe8d29b1cbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:45.345105 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-c30d0386-e4cd-4bff-865e-fe8d29b1cbdb None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:45.346390 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-c30d0386-e4cd-4bff-865e-fe8d29b1cbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:45.354172 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 691/1378] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:30:45.400118 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafc622-7a14-45d5-baf9-3f7f7d4f065e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:30:45.414590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafc622-7a14-45d5-baf9-3f7f7d4f065e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:30:45.419990 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c4c19-cd85-43bd-8f63-42a6da350c57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:30:45.420623 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c4c19-cd85-43bd-8f63-42a6da350c57 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:45.420905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c4c19-cd85-43bd-8f63-42a6da350c57 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:45.421184 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c4c19-cd85-43bd-8f63-42a6da350c57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:45.430083 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c4c19-cd85-43bd-8f63-42a6da350c57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:30:45.431661 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c4c19-cd85-43bd-8f63-42a6da350c57 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:30:45.442090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 688/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:30:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:30:47.766175 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd6292-040b-4019-8ebf-9f47a5f21581 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:47.766725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd6292-040b-4019-8ebf-9f47a5f21581 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:47.766725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd6292-040b-4019-8ebf-9f47a5f21581 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:47.769682 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 692/1380] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:30:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:47.773766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac42bf-71c2-4cf6-b485-e0633d6ac7f5 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:47.774173 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac42bf-71c2-4cf6-b485-e0633d6ac7f5 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:47.774323 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac42bf-71c2-4cf6-b485-e0633d6ac7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:47.791858 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ceac42bf-71c2-4cf6-b485-e0633d6ac7f5 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:47.793044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ceac42bf-71c2-4cf6-b485-e0633d6ac7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:47.801932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 689/1381] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:30:47.847365 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd6292-040b-4019-8ebf-9f47a5f21581 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:30:47.862537 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd6292-040b-4019-8ebf-9f47a5f21581 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:30:47.867961 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ad364-7152-44ec-8c2d-8180c1727f81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:30:47.868608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ad364-7152-44ec-8c2d-8180c1727f81 service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:47.868892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ad364-7152-44ec-8c2d-8180c1727f81 service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:47.869168 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ad364-7152-44ec-8c2d-8180c1727f81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:47.878320 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ad364-7152-44ec-8c2d-8180c1727f81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:30:47.880088 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ad364-7152-44ec-8c2d-8180c1727f81 service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:30:47.890863 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 693/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:30:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:30:50.286108 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5f731-89d5-4b42-b807-97e8ab7d4624 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:50.286593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5f731-89d5-4b42-b807-97e8ab7d4624 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:50.286703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5f731-89d5-4b42-b807-97e8ab7d4624 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:50.288218 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 690/1383] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:50.293506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811eb75-9d83-4719-8428-a89fe27bc164 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:50.294500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811eb75-9d83-4719-8428-a89fe27bc164 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:50.294778 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811eb75-9d83-4719-8428-a89fe27bc164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:50.312804 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-b811eb75-9d83-4719-8428-a89fe27bc164 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:50.314586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-b811eb75-9d83-4719-8428-a89fe27bc164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:50.322789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 694/1384] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:30:50.366561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5f731-89d5-4b42-b807-97e8ab7d4624 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:30:50.384242 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5f731-89d5-4b42-b807-97e8ab7d4624 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:30:50.392678 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bd11c-fd0a-400d-ad5b-00aaa826a5f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:30:50.394302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bd11c-fd0a-400d-ad5b-00aaa826a5f0 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:50.394773 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bd11c-fd0a-400d-ad5b-00aaa826a5f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:50.395108 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bd11c-fd0a-400d-ad5b-00aaa826a5f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:50.404104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387bd11c-fd0a-400d-ad5b-00aaa826a5f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:30:50.405860 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387bd11c-fd0a-400d-ad5b-00aaa826a5f0 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:30:50.418635 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 691/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:30:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:30:52.742997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712861c-9fdf-4175-8faf-864a26dd5f04 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:52.743546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712861c-9fdf-4175-8faf-864a26dd5f04 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:52.743546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712861c-9fdf-4175-8faf-864a26dd5f04 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:52.745002 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 695/1386] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:52.750390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66a9ac-5c11-4f3e-b430-81f239aa3a13 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:52.750800 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66a9ac-5c11-4f3e-b430-81f239aa3a13 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:52.750937 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66a9ac-5c11-4f3e-b430-81f239aa3a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:52.773536 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ad66a9ac-5c11-4f3e-b430-81f239aa3a13 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:52.775216 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ad66a9ac-5c11-4f3e-b430-81f239aa3a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:52.786000 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 692/1387] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:30:52.831181 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712861c-9fdf-4175-8faf-864a26dd5f04 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:30:52.849845 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712861c-9fdf-4175-8faf-864a26dd5f04 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:30:52.856772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b2005b-cbb1-414e-a3d1-ff2e39d5e222 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:30:52.857636 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2005b-cbb1-414e-a3d1-ff2e39d5e222 service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:52.857991 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2005b-cbb1-414e-a3d1-ff2e39d5e222 service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:52.858359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2005b-cbb1-414e-a3d1-ff2e39d5e222 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:52.870176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b2005b-cbb1-414e-a3d1-ff2e39d5e222 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:30:52.872295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b2005b-cbb1-414e-a3d1-ff2e39d5e222 service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:30:52.885150 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 696/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:30:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:30:55.198546 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b9e05-af53-4ddd-b2d6-62d384916920 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:55.199248 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b9e05-af53-4ddd-b2d6-62d384916920 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:55.199248 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b9e05-af53-4ddd-b2d6-62d384916920 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:55.200488 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 693/1389] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:55.206429 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28da77-1cfc-464e-a760-137dd4dfd059 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:55.206875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28da77-1cfc-464e-a760-137dd4dfd059 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:55.206992 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28da77-1cfc-464e-a760-137dd4dfd059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:55.225215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-3b28da77-1cfc-464e-a760-137dd4dfd059 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:55.226666 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-3b28da77-1cfc-464e-a760-137dd4dfd059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:55.236554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 697/1390] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:30:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:30:55.282258 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b9e05-af53-4ddd-b2d6-62d384916920 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:30:55.296572 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b9e05-af53-4ddd-b2d6-62d384916920 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:30:55.302598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e75a2-57d9-47c8-8e75-d30e3eb5dd7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:30:55.303236 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e75a2-57d9-47c8-8e75-d30e3eb5dd7a service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:55.303515 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e75a2-57d9-47c8-8e75-d30e3eb5dd7a service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:55.303812 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e75a2-57d9-47c8-8e75-d30e3eb5dd7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:55.312153 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e75a2-57d9-47c8-8e75-d30e3eb5dd7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:30:55.313802 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e75a2-57d9-47c8-8e75-d30e3eb5dd7a service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:30:55.323567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 694/1391] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:30:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:30:57.670833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa899a2-b498-47e0-94de-c4eb91ec37a2 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:57.671413 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa899a2-b498-47e0-94de-c4eb91ec37a2 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:57.671413 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa899a2-b498-47e0-94de-c4eb91ec37a2 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:57.672872 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 698/1392] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:57.678383 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cff9a-2fb5-4dd6-bfcc-d708f60755bc None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:57.678845 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cff9a-2fb5-4dd6-bfcc-d708f60755bc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:57.678950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cff9a-2fb5-4dd6-bfcc-d708f60755bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:57.697079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-610cff9a-2fb5-4dd6-bfcc-d708f60755bc None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:57.698283 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-610cff9a-2fb5-4dd6-bfcc-d708f60755bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:57.711033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 695/1393] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:30:57.759794 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa899a2-b498-47e0-94de-c4eb91ec37a2 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:30:57.774831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa899a2-b498-47e0-94de-c4eb91ec37a2 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:30:57.780306 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f46ee-d6ef-4b16-9d68-b9633271ce87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:30:57.780949 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f46ee-d6ef-4b16-9d68-b9633271ce87 service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:57.781236 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f46ee-d6ef-4b16-9d68-b9633271ce87 service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:57.781514 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f46ee-d6ef-4b16-9d68-b9633271ce87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:57.790218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f46ee-d6ef-4b16-9d68-b9633271ce87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:30:57.791928 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f46ee-d6ef-4b16-9d68-b9633271ce87 service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:30:57.801869 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 699/1394] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:30:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:30:59.762791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb19ce-7f94-41f6-adeb-33800ce70212 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:59.763365 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb19ce-7f94-41f6-adeb-33800ce70212 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:59.763365 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb19ce-7f94-41f6-adeb-33800ce70212 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:59.764794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 696/1395] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:59.770945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68d478-21e1-416b-8a43-5630b35e869e None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:59.771407 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68d478-21e1-416b-8a43-5630b35e869e None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:59.771568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68d478-21e1-416b-8a43-5630b35e869e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:59.778546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=52 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:30:59.792072 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-cb68d478-21e1-416b-8a43-5630b35e869e None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:59.793255 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-cb68d478-21e1-416b-8a43-5630b35e869e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:30:59.801432 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 700/1396] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:30:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:30:59.859418 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5094bd8-181c-4ab4-8319-f25198e8b143 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:59.859737 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5094bd8-181c-4ab4-8319-f25198e8b143 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:59.860012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5094bd8-181c-4ab4-8319-f25198e8b143 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:59.861569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 697/1397] 10.222.0.42 () {54 vars in 962 bytes} [Mon Jun 6 10:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:30:59.866910 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef624a-5561-4f95-acc6-c1a8a96b9dd8 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:30:59.867283 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef624a-5561-4f95-acc6-c1a8a96b9dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:30:59.867625 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef624a-5561-4f95-acc6-c1a8a96b9dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:30:59.917435 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-eaef624a-5561-4f95-acc6-c1a8a96b9dd8 None None] MFA Rules not processed for user `1a67e007fc314b34aac2471a7c20f240`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:30:59.918620 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-eaef624a-5561-4f95-acc6-c1a8a96b9dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:00.109850 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 701/1398] 10.222.0.42 () {58 vars in 1059 bytes} [Mon Jun 6 10:30:59 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:00.117979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5094bd8-181c-4ab4-8319-f25198e8b143 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:00.129227 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5094bd8-181c-4ab4-8319-f25198e8b143 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:00.132810 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988108d8-b657-4797-a233-4dc787e899af service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a67e007fc314b34aac2471a7c20f240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:00.133448 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988108d8-b657-4797-a233-4dc787e899af service glance] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:00.133734 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988108d8-b657-4797-a233-4dc787e899af service glance] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:00.134013 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988108d8-b657-4797-a233-4dc787e899af service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:00.144079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988108d8-b657-4797-a233-4dc787e899af service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:00.145901 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988108d8-b657-4797-a233-4dc787e899af service glance] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:00.155492 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 698/1399] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:31:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:31:00.176272 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72082118-8fe9-4a19-9d57-4fd7e28a7965 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:00.176561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72082118-8fe9-4a19-9d57-4fd7e28a7965 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:00.176841 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72082118-8fe9-4a19-9d57-4fd7e28a7965 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:00.178268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 702/1400] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:00.183446 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b92a5e-ee0b-404b-bf77-268742832ec1 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:00.183835 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b92a5e-ee0b-404b-bf77-268742832ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:00.184184 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b92a5e-ee0b-404b-bf77-268742832ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:00.202219 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-22b92a5e-ee0b-404b-bf77-268742832ec1 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:00.203404 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-22b92a5e-ee0b-404b-bf77-268742832ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:00.211821 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 699/1401] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:00.258150 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72082118-8fe9-4a19-9d57-4fd7e28a7965 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:00.272919 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72082118-8fe9-4a19-9d57-4fd7e28a7965 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:00.278395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed32fa7-46c3-4dc7-ab81-a54a1d52c2f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:00.279030 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed32fa7-46c3-4dc7-ab81-a54a1d52c2f9 service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:00.279309 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed32fa7-46c3-4dc7-ab81-a54a1d52c2f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:00.279584 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed32fa7-46c3-4dc7-ab81-a54a1d52c2f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:00.288118 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed32fa7-46c3-4dc7-ab81-a54a1d52c2f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:00.289891 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed32fa7-46c3-4dc7-ab81-a54a1d52c2f9 service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:00.300252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 703/1402] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:31:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:31:00.696858 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c17d1f-2558-4d26-aa48-17e5a562947e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:00.697146 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c17d1f-2558-4d26-aa48-17e5a562947e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:00.697423 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c17d1f-2558-4d26-aa48-17e5a562947e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:00.745471 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-00c17d1f-2558-4d26-aa48-17e5a562947e None None] MFA Rules not processed for user `e9e358afe0974c0c8bd53104ebbff445`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:00.746654 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-00c17d1f-2558-4d26-aa48-17e5a562947e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:00.778816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:31:00.817087 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 700/1403] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jun 6 10:31:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:00.864884 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64197a18-5e5b-44a3-b5d8-96cd6795ec7c None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:00.865393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64197a18-5e5b-44a3-b5d8-96cd6795ec7c None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:00.865597 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64197a18-5e5b-44a3-b5d8-96cd6795ec7c None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:00.867168 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 704/1404] 10.222.0.42 () {54 vars in 942 bytes} [Mon Jun 6 10:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:00.873521 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3739414-edcf-47af-a0c3-ab4de6bc9ecc None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:00.873819 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3739414-edcf-47af-a0c3-ab4de6bc9ecc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:00.874107 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3739414-edcf-47af-a0c3-ab4de6bc9ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:00.923883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c3739414-edcf-47af-a0c3-ab4de6bc9ecc None None] MFA Rules not processed for user `fede9e52ad274285b09d748a684ede38`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:00.925076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c3739414-edcf-47af-a0c3-ab4de6bc9ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:00.996094 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 701/1405] 10.222.0.42 () {58 vars in 1039 bytes} [Mon Jun 6 10:31:00 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:01.006173 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64197a18-5e5b-44a3-b5d8-96cd6795ec7c None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:01.020268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64197a18-5e5b-44a3-b5d8-96cd6795ec7c None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:01.025083 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95197925-4fc6-4540-b27c-27501490236d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fede9e52ad274285b09d748a684ede38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:01.025712 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95197925-4fc6-4540-b27c-27501490236d service swift] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:01.025996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95197925-4fc6-4540-b27c-27501490236d service swift] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:01.026266 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95197925-4fc6-4540-b27c-27501490236d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:01.035079 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95197925-4fc6-4540-b27c-27501490236d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:01.036699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95197925-4fc6-4540-b27c-27501490236d service swift] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:01.044677 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 705/1406] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jun 6 10:31:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 10:31:02.648320 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3325b9-cdc9-46b6-bd8a-fd4fd428dd98 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:02.648866 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3325b9-cdc9-46b6-bd8a-fd4fd428dd98 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:02.648866 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3325b9-cdc9-46b6-bd8a-fd4fd428dd98 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:02.650122 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 702/1407] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:31:02.655484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2dc78-53a9-4ac1-a5d0-bb307d7f3626 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:02.655944 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2dc78-53a9-4ac1-a5d0-bb307d7f3626 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:02.656197 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2dc78-53a9-4ac1-a5d0-bb307d7f3626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:02.675068 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-fac2dc78-53a9-4ac1-a5d0-bb307d7f3626 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:02.676286 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-fac2dc78-53a9-4ac1-a5d0-bb307d7f3626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:02.684287 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 706/1408] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:31:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:02.729720 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3325b9-cdc9-46b6-bd8a-fd4fd428dd98 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:02.745668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3325b9-cdc9-46b6-bd8a-fd4fd428dd98 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:02.751723 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64c0ea-b4a8-4798-a887-a96d2be944f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:02.752719 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64c0ea-b4a8-4798-a887-a96d2be944f2 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:02.752998 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64c0ea-b4a8-4798-a887-a96d2be944f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:02.753272 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64c0ea-b4a8-4798-a887-a96d2be944f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:02.761666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64c0ea-b4a8-4798-a887-a96d2be944f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:02.763250 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64c0ea-b4a8-4798-a887-a96d2be944f2 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:02.775953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 703/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:31:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:31:12.669710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=55 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:31:12.769580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=62 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:31:24.842440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23842ecc-6a2e-40e3-a6f8-e5d5b88a7d26 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:24.842904 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23842ecc-6a2e-40e3-a6f8-e5d5b88a7d26 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:24.843009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23842ecc-6a2e-40e3-a6f8-e5d5b88a7d26 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:24.844367 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 707/1410] 10.222.0.42 () {54 vars in 918 bytes} [Mon Jun 6 10:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:24.849677 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de66efb-989b-45fe-b511-812deb322b5e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:24.850176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de66efb-989b-45fe-b511-812deb322b5e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:24.850304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de66efb-989b-45fe-b511-812deb322b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:24.867726 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-7de66efb-989b-45fe-b511-812deb322b5e None None] MFA Rules not processed for user `361be69f1c0a41c18afd206bccf56447`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:24.868930 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-7de66efb-989b-45fe-b511-812deb322b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:24.877916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 704/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Mon Jun 6 10:31:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:24.891742 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef69fd-a54f-4e56-9bbd-5f80616d3faf None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:24.892593 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef69fd-a54f-4e56-9bbd-5f80616d3faf None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:24.892869 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef69fd-a54f-4e56-9bbd-5f80616d3faf None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:24.894203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 708/1412] 10.222.0.42 () {54 vars in 918 bytes} [Mon Jun 6 10:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:24.898810 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67065fc3-732b-4f86-94b4-3310fed9c3ac None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:24.899100 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67065fc3-732b-4f86-94b4-3310fed9c3ac None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:24.899377 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67065fc3-732b-4f86-94b4-3310fed9c3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:24.916899 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-67065fc3-732b-4f86-94b4-3310fed9c3ac None None] MFA Rules not processed for user `ed8a40e4f1034a8e99d0eb505e448db9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:24.918128 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-67065fc3-732b-4f86-94b4-3310fed9c3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:24.927644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 705/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Mon Jun 6 10:31:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:31:27.692490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b2437-07f0-4210-bf62-4a6500937bde None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:27.693053 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b2437-07f0-4210-bf62-4a6500937bde None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:27.693053 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b2437-07f0-4210-bf62-4a6500937bde None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:27.694420 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 10:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:27.699607 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704642f-4b8d-43e0-9865-5c6a89f48f14 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:27.700100 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704642f-4b8d-43e0-9865-5c6a89f48f14 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:27.700215 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704642f-4b8d-43e0-9865-5c6a89f48f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:27.718005 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3704642f-4b8d-43e0-9865-5c6a89f48f14 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:27.719194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3704642f-4b8d-43e0-9865-5c6a89f48f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:27.726960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 10:31:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:27.772244 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b2437-07f0-4210-bf62-4a6500937bde None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:27.788524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b2437-07f0-4210-bf62-4a6500937bde None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:27.793690 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfff990-ecbc-4494-af0f-435f7da183b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:27.794312 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfff990-ecbc-4494-af0f-435f7da183b7 service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:27.794587 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfff990-ecbc-4494-af0f-435f7da183b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:27.794854 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfff990-ecbc-4494-af0f-435f7da183b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:27.803144 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfff990-ecbc-4494-af0f-435f7da183b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:27.804714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfff990-ecbc-4494-af0f-435f7da183b7 service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:27.814655 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 710/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:31:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:31:28.806177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fc995-9fdb-408c-a00f-fb2cd920bd4a None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:28.806743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fc995-9fdb-408c-a00f-fb2cd920bd4a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:28.806743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fc995-9fdb-408c-a00f-fb2cd920bd4a None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:28.808114 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 10:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:28.812953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b43391-911a-4fc5-9f34-391aa332a3e9 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:28.813346 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b43391-911a-4fc5-9f34-391aa332a3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:28.813512 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b43391-911a-4fc5-9f34-391aa332a3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:28.831347 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-f5b43391-911a-4fc5-9f34-391aa332a3e9 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:28.832544 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-f5b43391-911a-4fc5-9f34-391aa332a3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:28.840216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 10:31:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:29.013006 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fc995-9fdb-408c-a00f-fb2cd920bd4a None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:29.027584 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fc995-9fdb-408c-a00f-fb2cd920bd4a None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:29.032984 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3809ec5-f845-40a4-aca8-cb05915dcca9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:29.033642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3809ec5-f845-40a4-aca8-cb05915dcca9 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:29.033924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3809ec5-f845-40a4-aca8-cb05915dcca9 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:29.034198 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3809ec5-f845-40a4-aca8-cb05915dcca9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:29.042225 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3809ec5-f845-40a4-aca8-cb05915dcca9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:29.043804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3809ec5-f845-40a4-aca8-cb05915dcca9 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:29.053626 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 708/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:31:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:31:29.987675 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b8362-1201-4450-bd85-f972251002f0 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:29.988243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b8362-1201-4450-bd85-f972251002f0 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:29.988243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b8362-1201-4450-bd85-f972251002f0 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:29.989616 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 10:31:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:29.994495 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4e4e9-a988-4314-b8b0-d1c7f5b5224d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:29.994943 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4e4e9-a988-4314-b8b0-d1c7f5b5224d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:29.995049 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4e4e9-a988-4314-b8b0-d1c7f5b5224d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:30.012442 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-89e4e4e9-a988-4314-b8b0-d1c7f5b5224d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:30.013631 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-89e4e4e9-a988-4314-b8b0-d1c7f5b5224d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:30.021830 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 10:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:31:30.170282 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b8362-1201-4450-bd85-f972251002f0 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:30.185108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b8362-1201-4450-bd85-f972251002f0 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:30.190638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01f4b9-e032-49d5-96a2-6521fc6daac6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:30.191266 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01f4b9-e032-49d5-96a2-6521fc6daac6 service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:30.191545 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01f4b9-e032-49d5-96a2-6521fc6daac6 service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:30.191848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01f4b9-e032-49d5-96a2-6521fc6daac6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:30.199944 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c01f4b9-e032-49d5-96a2-6521fc6daac6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:30.201499 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c01f4b9-e032-49d5-96a2-6521fc6daac6 service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:30.211716 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 713/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:31:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:31:31.124782 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29d18b-3a15-4fbc-96b7-82ab0c5e94e0 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:31.125311 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29d18b-3a15-4fbc-96b7-82ab0c5e94e0 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:31.125311 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29d18b-3a15-4fbc-96b7-82ab0c5e94e0 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:31.127601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 10:31:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:31.132581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e54a0-57ec-41b4-adac-457545c859fd None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:31.133009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e54a0-57ec-41b4-adac-457545c859fd None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:31.133121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e54a0-57ec-41b4-adac-457545c859fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:31.151004 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-cc4e54a0-57ec-41b4-adac-457545c859fd None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:31.152213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-cc4e54a0-57ec-41b4-adac-457545c859fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:31.160028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 10:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:31.313530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29d18b-3a15-4fbc-96b7-82ab0c5e94e0 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:31.328283 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29d18b-3a15-4fbc-96b7-82ab0c5e94e0 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:31.333653 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587410ac-fb6a-4451-a992-42a70e0f7c42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:31.334288 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587410ac-fb6a-4451-a992-42a70e0f7c42 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:31.334567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587410ac-fb6a-4451-a992-42a70e0f7c42 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:31.334841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587410ac-fb6a-4451-a992-42a70e0f7c42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:31.343536 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587410ac-fb6a-4451-a992-42a70e0f7c42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:31.345641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587410ac-fb6a-4451-a992-42a70e0f7c42 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:31.354780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 711/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:31:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:31:32.263933 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a6896-e408-4a73-aaeb-09702355550a None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:32.264473 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a6896-e408-4a73-aaeb-09702355550a None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:32.264473 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a6896-e408-4a73-aaeb-09702355550a None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:32.265892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 10:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:32.271064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e969151-e011-4ce0-9d45-7135bb2acdae None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:32.271489 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e969151-e011-4ce0-9d45-7135bb2acdae None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:32.271668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e969151-e011-4ce0-9d45-7135bb2acdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:32.289922 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-1e969151-e011-4ce0-9d45-7135bb2acdae None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:32.291093 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1e969151-e011-4ce0-9d45-7135bb2acdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:32.299298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 10:31:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:32.451725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a6896-e408-4a73-aaeb-09702355550a None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:32.466336 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a6896-e408-4a73-aaeb-09702355550a None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:32.472314 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994926b5-b7fc-4b43-ad67-84c06fdbe3fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:32.472939 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994926b5-b7fc-4b43-ad67-84c06fdbe3fc service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:32.473234 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994926b5-b7fc-4b43-ad67-84c06fdbe3fc service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:32.473488 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994926b5-b7fc-4b43-ad67-84c06fdbe3fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:32.481409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994926b5-b7fc-4b43-ad67-84c06fdbe3fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:32.482957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994926b5-b7fc-4b43-ad67-84c06fdbe3fc service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:32.492304 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 716/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:31:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:31:33.347920 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbc94b-665d-439b-bad4-b1c91b73a5ee None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:33.348487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbc94b-665d-439b-bad4-b1c91b73a5ee None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:33.348487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbc94b-665d-439b-bad4-b1c91b73a5ee None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:33.349884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 10:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:33.354776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372bb57-5f18-465c-a798-8ab75331331d None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:33.355206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372bb57-5f18-465c-a798-8ab75331331d None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:33.355316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372bb57-5f18-465c-a798-8ab75331331d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:33.373386 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-2372bb57-5f18-465c-a798-8ab75331331d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:33.374572 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-2372bb57-5f18-465c-a798-8ab75331331d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:33.383884 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 10:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:33.536902 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffbc94b-665d-439b-bad4-b1c91b73a5ee None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:33.551728 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffbc94b-665d-439b-bad4-b1c91b73a5ee None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:33.557039 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19964a-8b66-430c-b9db-ef5e2b8b9da9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:33.557668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19964a-8b66-430c-b9db-ef5e2b8b9da9 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:33.557952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19964a-8b66-430c-b9db-ef5e2b8b9da9 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:33.558227 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19964a-8b66-430c-b9db-ef5e2b8b9da9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:33.566889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc19964a-8b66-430c-b9db-ef5e2b8b9da9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:33.568502 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc19964a-8b66-430c-b9db-ef5e2b8b9da9 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:33.577720 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 714/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:31:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:31:34.434806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615d0cf-95d9-4ff6-9b61-dc0ed1f614e1 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:34.435327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615d0cf-95d9-4ff6-9b61-dc0ed1f614e1 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:34.435424 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615d0cf-95d9-4ff6-9b61-dc0ed1f614e1 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:34.436740 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 10:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:34.441665 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374effeb-4ecc-4631-af46-d3c68aa76868 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:34.442059 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374effeb-4ecc-4631-af46-d3c68aa76868 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:34.442212 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374effeb-4ecc-4631-af46-d3c68aa76868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:34.459662 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-374effeb-4ecc-4631-af46-d3c68aa76868 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:34.460936 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-374effeb-4ecc-4631-af46-d3c68aa76868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:34.468478 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 10:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:34.621112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615d0cf-95d9-4ff6-9b61-dc0ed1f614e1 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:34.634978 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615d0cf-95d9-4ff6-9b61-dc0ed1f614e1 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:34.640316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d882615-bdf3-48d7-b523-409090bfb75d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:34.640884 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d882615-bdf3-48d7-b523-409090bfb75d service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:34.641163 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d882615-bdf3-48d7-b523-409090bfb75d service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:34.641432 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d882615-bdf3-48d7-b523-409090bfb75d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:34.649701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d882615-bdf3-48d7-b523-409090bfb75d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:34.651246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d882615-bdf3-48d7-b523-409090bfb75d service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:34.660998 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 719/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:31:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:31:35.522092 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcfa0b-b00f-47df-9fba-162e43c03966 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:35.522618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcfa0b-b00f-47df-9fba-162e43c03966 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:35.522618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcfa0b-b00f-47df-9fba-162e43c03966 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:35.524030 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 10:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:35.529031 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103c653-0dc3-408b-b13d-465335f6a108 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:35.529442 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103c653-0dc3-408b-b13d-465335f6a108 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:35.529580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103c653-0dc3-408b-b13d-465335f6a108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:35.548192 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-b103c653-0dc3-408b-b13d-465335f6a108 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:35.549388 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-b103c653-0dc3-408b-b13d-465335f6a108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:35.557398 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 10:31:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:35.706342 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcfa0b-b00f-47df-9fba-162e43c03966 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:35.721137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcfa0b-b00f-47df-9fba-162e43c03966 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:35.726393 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d536fbb-305b-4bfe-9e28-96757e634318 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:35.727013 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d536fbb-305b-4bfe-9e28-96757e634318 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:35.727294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d536fbb-305b-4bfe-9e28-96757e634318 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:35.727583 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d536fbb-305b-4bfe-9e28-96757e634318 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:35.735516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d536fbb-305b-4bfe-9e28-96757e634318 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:35.737376 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d536fbb-305b-4bfe-9e28-96757e634318 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:35.746559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 717/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:31:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:31:36.620036 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de339064-ac26-4784-a460-526b55b7f5d7 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:36.620628 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de339064-ac26-4784-a460-526b55b7f5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:36.620628 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de339064-ac26-4784-a460-526b55b7f5d7 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:36.621924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 10:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:36.626963 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de788f7-b902-4c36-a438-24e9e87954bd None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:36.627372 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de788f7-b902-4c36-a438-24e9e87954bd None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:36.627529 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de788f7-b902-4c36-a438-24e9e87954bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:36.645372 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3de788f7-b902-4c36-a438-24e9e87954bd None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:36.646751 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3de788f7-b902-4c36-a438-24e9e87954bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:36.654790 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 10:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:36.808288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de339064-ac26-4784-a460-526b55b7f5d7 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:36.822575 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de339064-ac26-4784-a460-526b55b7f5d7 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:36.827802 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fce1f0-1062-4f00-a2e0-a83c68d9352d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:36.828425 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fce1f0-1062-4f00-a2e0-a83c68d9352d service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:36.828705 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fce1f0-1062-4f00-a2e0-a83c68d9352d service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:36.828972 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fce1f0-1062-4f00-a2e0-a83c68d9352d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:36.837480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fce1f0-1062-4f00-a2e0-a83c68d9352d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:36.839072 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fce1f0-1062-4f00-a2e0-a83c68d9352d service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:36.849243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 722/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:31:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:31:38.985837 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8af65-f02b-4bab-b3ad-92182d117fd7 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:38.986396 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8af65-f02b-4bab-b3ad-92182d117fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:38.986500 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8af65-f02b-4bab-b3ad-92182d117fd7 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:38.987853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 10:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:38.993186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011e404-607c-4191-952c-22e83040e4e1 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:38.993587 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011e404-607c-4191-952c-22e83040e4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:38.993735 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011e404-607c-4191-952c-22e83040e4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:39.016119 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-b011e404-607c-4191-952c-22e83040e4e1 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:39.017927 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-b011e404-607c-4191-952c-22e83040e4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:39.025964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 10:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:39.064238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8af65-f02b-4bab-b3ad-92182d117fd7 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:39.074885 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8af65-f02b-4bab-b3ad-92182d117fd7 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:39.078337 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa49d54-8f02-4a76-98f8-788d956ce0cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:39.078966 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa49d54-8f02-4a76-98f8-788d956ce0cf service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:39.079867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa49d54-8f02-4a76-98f8-788d956ce0cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:39.080141 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa49d54-8f02-4a76-98f8-788d956ce0cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:39.089164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa49d54-8f02-4a76-98f8-788d956ce0cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:39.090608 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa49d54-8f02-4a76-98f8-788d956ce0cf service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:39.100014 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 720/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:31:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:31:41.496663 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7122f-73d0-4f11-9ac5-0a7efd10753a None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:41.497215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7122f-73d0-4f11-9ac5-0a7efd10753a None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:41.497215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7122f-73d0-4f11-9ac5-0a7efd10753a None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:41.498573 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 10:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:41.503942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8ef47-bc66-452e-8a7b-b7d46c03d200 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:41.504392 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8ef47-bc66-452e-8a7b-b7d46c03d200 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:41.504617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8ef47-bc66-452e-8a7b-b7d46c03d200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:41.522797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2ab8ef47-bc66-452e-8a7b-b7d46c03d200 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:41.524008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2ab8ef47-bc66-452e-8a7b-b7d46c03d200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:41.531771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 10:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:41.570014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7122f-73d0-4f11-9ac5-0a7efd10753a None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:41.571680 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-31e7122f-73d0-4f11-9ac5-0a7efd10753a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:41.584843 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7122f-73d0-4f11-9ac5-0a7efd10753a None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:41.588294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81204bd-be52-4545-8660-022dc95b7c16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:41.588921 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81204bd-be52-4545-8660-022dc95b7c16 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:41.589201 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81204bd-be52-4545-8660-022dc95b7c16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:41.589476 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81204bd-be52-4545-8660-022dc95b7c16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:41.599032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81204bd-be52-4545-8660-022dc95b7c16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:41.600465 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81204bd-be52-4545-8660-022dc95b7c16 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:41.609897 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 725/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:31:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:31:44.203681 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aceb9d7-42a9-41f2-81c8-ee3713f10651 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:44.204922 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aceb9d7-42a9-41f2-81c8-ee3713f10651 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:44.204922 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aceb9d7-42a9-41f2-81c8-ee3713f10651 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:44.206462 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 10:31:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:44.211774 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcaefd-651b-48f9-a8bc-8564e5235cb2 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:44.212253 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcaefd-651b-48f9-a8bc-8564e5235cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:44.212364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcaefd-651b-48f9-a8bc-8564e5235cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:44.229871 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-c7bcaefd-651b-48f9-a8bc-8564e5235cb2 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:44.231063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-c7bcaefd-651b-48f9-a8bc-8564e5235cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:44.239114 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 10:31:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:44.278533 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aceb9d7-42a9-41f2-81c8-ee3713f10651 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:44.289159 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aceb9d7-42a9-41f2-81c8-ee3713f10651 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:44.292613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835abfb7-fe5c-4804-978b-01b28d9682c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:44.293243 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835abfb7-fe5c-4804-978b-01b28d9682c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:44.293526 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835abfb7-fe5c-4804-978b-01b28d9682c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:44.293802 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835abfb7-fe5c-4804-978b-01b28d9682c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:44.302519 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835abfb7-fe5c-4804-978b-01b28d9682c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:44.303962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835abfb7-fe5c-4804-978b-01b28d9682c0 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:44.313233 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 723/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:31:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:31:46.666911 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c5a47-e66c-4a2a-a08d-88c5d62db0c7 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:46.667456 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c5a47-e66c-4a2a-a08d-88c5d62db0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:46.667456 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c5a47-e66c-4a2a-a08d-88c5d62db0c7 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:46.669096 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 10:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:46.674108 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730167f0-ef87-4390-9844-d11d2e34d411 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:46.674577 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730167f0-ef87-4390-9844-d11d2e34d411 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:46.674688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730167f0-ef87-4390-9844-d11d2e34d411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:46.692496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-730167f0-ef87-4390-9844-d11d2e34d411 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:46.693679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-730167f0-ef87-4390-9844-d11d2e34d411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:46.701353 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 10:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:46.740711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c5a47-e66c-4a2a-a08d-88c5d62db0c7 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:46.751729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c5a47-e66c-4a2a-a08d-88c5d62db0c7 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:46.755487 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33492b-7fd6-4dc9-a376-768f9a454def service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:46.756186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33492b-7fd6-4dc9-a376-768f9a454def service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:46.756467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33492b-7fd6-4dc9-a376-768f9a454def service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:46.756739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33492b-7fd6-4dc9-a376-768f9a454def service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:46.766299 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a33492b-7fd6-4dc9-a376-768f9a454def service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:46.767714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a33492b-7fd6-4dc9-a376-768f9a454def service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:46.777705 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 728/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:31:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:31:49.084959 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e9559-59ee-4642-90ac-b16b64020153 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:49.085537 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e9559-59ee-4642-90ac-b16b64020153 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:49.085537 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e9559-59ee-4642-90ac-b16b64020153 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:49.086869 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 10:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:49.092222 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc382870-39bc-4fba-aee2-d8adc80baa58 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:49.092701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc382870-39bc-4fba-aee2-d8adc80baa58 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:49.092811 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc382870-39bc-4fba-aee2-d8adc80baa58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:49.110820 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-bc382870-39bc-4fba-aee2-d8adc80baa58 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:49.112252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-bc382870-39bc-4fba-aee2-d8adc80baa58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:49.119873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 10:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:31:49.157923 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e9559-59ee-4642-90ac-b16b64020153 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:49.168445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e9559-59ee-4642-90ac-b16b64020153 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:49.171923 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ad919-8caa-4d3a-ba1a-4dd077ebdf43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:49.172553 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ad919-8caa-4d3a-ba1a-4dd077ebdf43 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:49.173408 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ad919-8caa-4d3a-ba1a-4dd077ebdf43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:49.173702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ad919-8caa-4d3a-ba1a-4dd077ebdf43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:49.182386 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ad919-8caa-4d3a-ba1a-4dd077ebdf43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:49.183814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ad919-8caa-4d3a-ba1a-4dd077ebdf43 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:49.193752 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 726/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:31:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:31:51.512774 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653c824-4a3f-4870-a53a-4019508a63c0 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:51.513352 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653c824-4a3f-4870-a53a-4019508a63c0 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:51.513352 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653c824-4a3f-4870-a53a-4019508a63c0 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:51.514687 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 10:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:51.520090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f21fc-f7d7-4ee8-b2df-d119970dda24 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:51.520540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f21fc-f7d7-4ee8-b2df-d119970dda24 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:51.520651 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f21fc-f7d7-4ee8-b2df-d119970dda24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:51.538457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-304f21fc-f7d7-4ee8-b2df-d119970dda24 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:51.539647 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-304f21fc-f7d7-4ee8-b2df-d119970dda24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:51.547307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 10:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:51.585233 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653c824-4a3f-4870-a53a-4019508a63c0 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:51.596293 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653c824-4a3f-4870-a53a-4019508a63c0 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:51.599832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66fef7-c6c9-4c3d-9d0e-e9a9c7dce49f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:51.600498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66fef7-c6c9-4c3d-9d0e-e9a9c7dce49f service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:51.600785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66fef7-c6c9-4c3d-9d0e-e9a9c7dce49f service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:51.601062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66fef7-c6c9-4c3d-9d0e-e9a9c7dce49f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:51.609760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66fef7-c6c9-4c3d-9d0e-e9a9c7dce49f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:51.611768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66fef7-c6c9-4c3d-9d0e-e9a9c7dce49f service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:51.620996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 731/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:31:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:31:53.997738 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c924d-b3fe-4b79-a1ab-9765eced43ab None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:53.998306 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c924d-b3fe-4b79-a1ab-9765eced43ab None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:53.998306 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c924d-b3fe-4b79-a1ab-9765eced43ab None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:53.999638 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 10:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:54.005786 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71bac9-239a-4137-af32-1813ccc6ba39 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:54.006217 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71bac9-239a-4137-af32-1813ccc6ba39 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:54.006317 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71bac9-239a-4137-af32-1813ccc6ba39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:54.012252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=70 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:31:54.025907 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-9c71bac9-239a-4137-af32-1813ccc6ba39 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:54.027081 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-9c71bac9-239a-4137-af32-1813ccc6ba39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:54.035165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 10:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:54.072712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c924d-b3fe-4b79-a1ab-9765eced43ab None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:54.078667 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=73 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:31:54.086317 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c924d-b3fe-4b79-a1ab-9765eced43ab None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:54.089843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520835d5-27cf-42d8-be52-29f6431038ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:54.090472 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520835d5-27cf-42d8-be52-29f6431038ac service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:54.090753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520835d5-27cf-42d8-be52-29f6431038ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:54.091025 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520835d5-27cf-42d8-be52-29f6431038ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:54.100363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520835d5-27cf-42d8-be52-29f6431038ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:54.101771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520835d5-27cf-42d8-be52-29f6431038ac service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:54.111624 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 729/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:31:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:31:56.442942 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2225a6-1409-43f8-bf67-114d6e78717f None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:56.443479 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2225a6-1409-43f8-bf67-114d6e78717f None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:56.443594 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2225a6-1409-43f8-bf67-114d6e78717f None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:56.444911 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 10:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:56.450276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67414e-3b33-4a14-bc95-d406a2b04b55 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:56.450678 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67414e-3b33-4a14-bc95-d406a2b04b55 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:56.450827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67414e-3b33-4a14-bc95-d406a2b04b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:56.469384 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-1d67414e-3b33-4a14-bc95-d406a2b04b55 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:56.470806 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1d67414e-3b33-4a14-bc95-d406a2b04b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:56.478660 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 10:31:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:56.516229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2225a6-1409-43f8-bf67-114d6e78717f None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:56.527260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2225a6-1409-43f8-bf67-114d6e78717f None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:56.530777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffdccec-e988-4962-b20b-ff14906eed9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:56.531405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdccec-e988-4962-b20b-ff14906eed9e service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:56.531684 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdccec-e988-4962-b20b-ff14906eed9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:56.532028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdccec-e988-4962-b20b-ff14906eed9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:56.540922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffdccec-e988-4962-b20b-ff14906eed9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:56.542326 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffdccec-e988-4962-b20b-ff14906eed9e service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:56.552294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 734/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:31:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:31:58.926518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7dc46-5800-43b2-bbe5-d1bb898a07c3 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:58.927083 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7dc46-5800-43b2-bbe5-d1bb898a07c3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:58.927083 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7dc46-5800-43b2-bbe5-d1bb898a07c3 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:58.928435 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 10:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:31:58.933607 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa4c92-5bc6-4ac8-8ba9-0a75b78957db None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:58.933995 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa4c92-5bc6-4ac8-8ba9-0a75b78957db None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:58.934162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa4c92-5bc6-4ac8-8ba9-0a75b78957db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:58.951476 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-e3fa4c92-5bc6-4ac8-8ba9-0a75b78957db None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:31:58.952691 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-e3fa4c92-5bc6-4ac8-8ba9-0a75b78957db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:31:58.961407 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 10:31:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:31:58.997686 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7dc46-5800-43b2-bbe5-d1bb898a07c3 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:31:59.008746 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7dc46-5800-43b2-bbe5-d1bb898a07c3 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:31:59.012264 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691b97a-e6b6-4d08-af4d-2ceac8ae8e86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:31:59.012892 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691b97a-e6b6-4d08-af4d-2ceac8ae8e86 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:31:59.013173 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691b97a-e6b6-4d08-af4d-2ceac8ae8e86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:31:59.013446 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691b97a-e6b6-4d08-af4d-2ceac8ae8e86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:31:59.022948 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691b97a-e6b6-4d08-af4d-2ceac8ae8e86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:31:59.024400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691b97a-e6b6-4d08-af4d-2ceac8ae8e86 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:31:59.034169 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 732/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:31:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:32:01.286213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d5134-7c2b-40b1-90fd-cb7ce1737d54 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:01.286745 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d5134-7c2b-40b1-90fd-cb7ce1737d54 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:01.286745 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d5134-7c2b-40b1-90fd-cb7ce1737d54 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:01.288177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 10:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:32:01.293536 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f26a1c-9395-4727-9331-451e30f447a6 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:01.293939 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f26a1c-9395-4727-9331-451e30f447a6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:01.294073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f26a1c-9395-4727-9331-451e30f447a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:01.311315 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-f2f26a1c-9395-4727-9331-451e30f447a6 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:32:01.312537 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f2f26a1c-9395-4727-9331-451e30f447a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:32:01.320418 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 10:32:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:32:01.357710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d5134-7c2b-40b1-90fd-cb7ce1737d54 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:32:01.368498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d5134-7c2b-40b1-90fd-cb7ce1737d54 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:32:01.372007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b791940-eea2-4f21-acc7-933e02d5346c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:32:01.372616 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b791940-eea2-4f21-acc7-933e02d5346c service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:01.372897 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b791940-eea2-4f21-acc7-933e02d5346c service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:01.373168 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b791940-eea2-4f21-acc7-933e02d5346c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:01.383113 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b791940-eea2-4f21-acc7-933e02d5346c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:32:01.384563 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b791940-eea2-4f21-acc7-933e02d5346c service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:32:01.394543 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 737/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:32:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:32:11.306254 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=17 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:32:11.389743 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=17 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:32:33.498792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10a769-5657-47d6-8bd8-0d0dcc608fd4 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:33.499378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10a769-5657-47d6-8bd8-0d0dcc608fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:33.499378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10a769-5657-47d6-8bd8-0d0dcc608fd4 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:33.501293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 734/1471] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:32:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:32:33.506540 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf9e19-5081-4fba-9e80-29219e9db578 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:33.506965 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf9e19-5081-4fba-9e80-29219e9db578 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:33.507097 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf9e19-5081-4fba-9e80-29219e9db578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:33.524949 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-31bf9e19-5081-4fba-9e80-29219e9db578 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:32:33.526135 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-31bf9e19-5081-4fba-9e80-29219e9db578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:32:33.533818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 738/1472] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:32:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:32:33.581125 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319fed4-643c-44ad-9be1-a2d0cf1c3550 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:33.581410 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319fed4-643c-44ad-9be1-a2d0cf1c3550 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:33.581685 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319fed4-643c-44ad-9be1-a2d0cf1c3550 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:33.583012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 735/1473] 10.222.0.42 () {54 vars in 962 bytes} [Mon Jun 6 10:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:32:33.587654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a1458-6502-4933-9afe-c8e56a6659e0 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:33.588045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a1458-6502-4933-9afe-c8e56a6659e0 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:33.588327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a1458-6502-4933-9afe-c8e56a6659e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:33.606865 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-dc1a1458-6502-4933-9afe-c8e56a6659e0 None None] MFA Rules not processed for user `1a67e007fc314b34aac2471a7c20f240`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:32:33.608069 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-dc1a1458-6502-4933-9afe-c8e56a6659e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:32:33.614956 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 739/1474] 10.222.0.42 () {58 vars in 1059 bytes} [Mon Jun 6 10:32:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:32:33.621996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319fed4-643c-44ad-9be1-a2d0cf1c3550 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:32:33.632346 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319fed4-643c-44ad-9be1-a2d0cf1c3550 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:32:33.635858 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9a622-165c-4705-92a6-ee6a86782bb8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a67e007fc314b34aac2471a7c20f240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:32:33.636411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9a622-165c-4705-92a6-ee6a86782bb8 service glance] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:33.636688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9a622-165c-4705-92a6-ee6a86782bb8 service glance] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:33.636958 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9a622-165c-4705-92a6-ee6a86782bb8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:33.645595 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9a622-165c-4705-92a6-ee6a86782bb8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:32:33.647253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9a622-165c-4705-92a6-ee6a86782bb8 service glance] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:32:33.656877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 736/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:32:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:32:36.043834 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d111352-6b67-4c15-84af-49cb25a1c3ff None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:36.044362 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d111352-6b67-4c15-84af-49cb25a1c3ff None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:36.044481 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d111352-6b67-4c15-84af-49cb25a1c3ff None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:36.045743 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 740/1476] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:32:36.051172 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243d20e-1fbd-4279-94c4-ecdc8846a9dd None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:36.051566 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243d20e-1fbd-4279-94c4-ecdc8846a9dd None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:36.051732 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243d20e-1fbd-4279-94c4-ecdc8846a9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:36.069620 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-4243d20e-1fbd-4279-94c4-ecdc8846a9dd None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:32:36.070813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4243d20e-1fbd-4279-94c4-ecdc8846a9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:32:36.079949 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 737/1477] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:32:36.103641 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d111352-6b67-4c15-84af-49cb25a1c3ff None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:32:36.116202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d111352-6b67-4c15-84af-49cb25a1c3ff None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:32:36.120605 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470da0c0-c669-44ba-8773-2ff61e0125f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:32:36.121260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470da0c0-c669-44ba-8773-2ff61e0125f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:36.121542 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470da0c0-c669-44ba-8773-2ff61e0125f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:36.121816 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470da0c0-c669-44ba-8773-2ff61e0125f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:36.122541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470da0c0-c669-44ba-8773-2ff61e0125f6 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:32:36.124328 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: WARNING py.warnings [None req-470da0c0-c669-44ba-8773-2ff61e0125f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 10:32:36.124328 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: warnings.warn(msg) Jun 06 10:32:36.124328 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: Jun 06 10:32:36.126223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470da0c0-c669-44ba-8773-2ff61e0125f6 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:32:36.145601 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 741/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jun 6 10:32:36 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:32:38.429277 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a1723-0d91-4631-9d11-2b9677db190d None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:38.429804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a1723-0d91-4631-9d11-2b9677db190d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:38.429804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a1723-0d91-4631-9d11-2b9677db190d None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:38.431173 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 738/1479] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:32:38.436519 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846e86a-cfe0-4ae2-918b-9e8d3999706e None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:38.436919 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846e86a-cfe0-4ae2-918b-9e8d3999706e None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:38.437061 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846e86a-cfe0-4ae2-918b-9e8d3999706e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:38.455789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-6846e86a-cfe0-4ae2-918b-9e8d3999706e None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:32:38.457014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-6846e86a-cfe0-4ae2-918b-9e8d3999706e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:32:38.464899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 742/1480] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:32:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:32:38.509679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a1723-0d91-4631-9d11-2b9677db190d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:32:38.524215 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a1723-0d91-4631-9d11-2b9677db190d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:32:38.529534 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd0edf-17b8-4511-81bb-410623a17acb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:32:38.530160 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd0edf-17b8-4511-81bb-410623a17acb service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:38.530443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd0edf-17b8-4511-81bb-410623a17acb service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:38.530714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd0edf-17b8-4511-81bb-410623a17acb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:38.538694 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd0edf-17b8-4511-81bb-410623a17acb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:32:38.540285 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd0edf-17b8-4511-81bb-410623a17acb service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:32:38.550047 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 739/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:32:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:32:41.011860 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ab32f-c668-4ea3-a18d-e6de4a339fcd None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:41.012436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ab32f-c668-4ea3-a18d-e6de4a339fcd None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:41.012436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ab32f-c668-4ea3-a18d-e6de4a339fcd None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:41.013781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 743/1482] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:32:41.019165 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f45974-9083-4846-bf5f-04b89f8c0866 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:41.019577 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f45974-9083-4846-bf5f-04b89f8c0866 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:41.019717 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f45974-9083-4846-bf5f-04b89f8c0866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:41.037885 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-21f45974-9083-4846-bf5f-04b89f8c0866 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:32:41.039069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-21f45974-9083-4846-bf5f-04b89f8c0866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:32:41.047382 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 740/1483] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:32:41.083891 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ab32f-c668-4ea3-a18d-e6de4a339fcd None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:32:41.095071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ab32f-c668-4ea3-a18d-e6de4a339fcd None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:32:41.098549 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abbae9b-489c-4191-b27f-5fcd4d864e77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:32:41.099176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbae9b-489c-4191-b27f-5fcd4d864e77 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:41.099460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbae9b-489c-4191-b27f-5fcd4d864e77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:41.099735 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbae9b-489c-4191-b27f-5fcd4d864e77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:41.108561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abbae9b-489c-4191-b27f-5fcd4d864e77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:32:41.110637 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abbae9b-489c-4191-b27f-5fcd4d864e77 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:32:41.119993 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 744/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:32:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:32:43.420743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbce457-70bb-4e10-87cc-25b1ea4587e9 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:43.421278 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbce457-70bb-4e10-87cc-25b1ea4587e9 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:43.421278 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbce457-70bb-4e10-87cc-25b1ea4587e9 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:43.422804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 741/1485] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:32:43.428009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a5c91-ae89-47f2-921c-8d46af0521d7 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:43.428421 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a5c91-ae89-47f2-921c-8d46af0521d7 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:43.428563 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a5c91-ae89-47f2-921c-8d46af0521d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:43.446567 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-411a5c91-ae89-47f2-921c-8d46af0521d7 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:32:43.447749 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-411a5c91-ae89-47f2-921c-8d46af0521d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:32:43.456162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 745/1486] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:32:43.493459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbce457-70bb-4e10-87cc-25b1ea4587e9 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:32:43.504851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbce457-70bb-4e10-87cc-25b1ea4587e9 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:32:43.508321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fcba57-c946-43b3-9238-3138bace935f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:32:43.508951 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fcba57-c946-43b3-9238-3138bace935f service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:43.509230 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fcba57-c946-43b3-9238-3138bace935f service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:43.509503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fcba57-c946-43b3-9238-3138bace935f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:43.519009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fcba57-c946-43b3-9238-3138bace935f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:32:43.520470 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fcba57-c946-43b3-9238-3138bace935f service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:32:43.530068 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 742/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:32:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:32:45.910655 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791c0d0-821d-4817-aa66-d1559f8a99a9 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:45.911305 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791c0d0-821d-4817-aa66-d1559f8a99a9 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:45.911305 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791c0d0-821d-4817-aa66-d1559f8a99a9 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:45.912632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 746/1488] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:32:45.917793 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19979a30-4382-4c77-98a4-9863b7d99dd2 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:45.918212 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19979a30-4382-4c77-98a4-9863b7d99dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:45.918355 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19979a30-4382-4c77-98a4-9863b7d99dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:45.936156 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-19979a30-4382-4c77-98a4-9863b7d99dd2 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:32:45.937484 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-19979a30-4382-4c77-98a4-9863b7d99dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:32:45.945279 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 743/1489] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:32:45.986323 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791c0d0-821d-4817-aa66-d1559f8a99a9 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:32:45.996898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791c0d0-821d-4817-aa66-d1559f8a99a9 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:32:46.000423 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03eff53-34a9-4194-8fca-707d81b928d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:32:46.001052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03eff53-34a9-4194-8fca-707d81b928d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:46.001338 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03eff53-34a9-4194-8fca-707d81b928d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:46.001612 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03eff53-34a9-4194-8fca-707d81b928d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:46.010431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03eff53-34a9-4194-8fca-707d81b928d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:32:46.011887 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03eff53-34a9-4194-8fca-707d81b928d9 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:32:46.021978 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 747/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:32:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:32:48.493960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cec93-9d7b-4a30-be06-ad5f91f75ffd None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:48.494517 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cec93-9d7b-4a30-be06-ad5f91f75ffd None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:48.494517 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cec93-9d7b-4a30-be06-ad5f91f75ffd None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:48.495876 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 744/1491] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jun 6 10:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:32:48.501629 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cc953-92e0-4974-bc99-adc701666300 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:48.502060 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cc953-92e0-4974-bc99-adc701666300 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:48.502212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cc953-92e0-4974-bc99-adc701666300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:48.519562 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-4f5cc953-92e0-4974-bc99-adc701666300 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:32:48.520794 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-4f5cc953-92e0-4974-bc99-adc701666300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:32:48.528779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 748/1492] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jun 6 10:32:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:32:48.568245 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6cec93-9d7b-4a30-be06-ad5f91f75ffd None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:32:48.578788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6cec93-9d7b-4a30-be06-ad5f91f75ffd None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:32:48.582256 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf20f68-f46e-4b92-8282-5462a7b30cb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:32:48.582881 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf20f68-f46e-4b92-8282-5462a7b30cb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:32:48.583162 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf20f68-f46e-4b92-8282-5462a7b30cb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:32:48.583435 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf20f68-f46e-4b92-8282-5462a7b30cb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:32:48.592099 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf20f68-f46e-4b92-8282-5462a7b30cb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:32:48.594105 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf20f68-f46e-4b92-8282-5462a7b30cb2 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:32:48.603345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 745/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:32:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:32:58.514384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=33 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:32:58.599278 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=33 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:34:21.485788 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b5038-0ef3-48ff-809c-a132d116a4e1 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:34:21.486370 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b5038-0ef3-48ff-809c-a132d116a4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:34:21.486370 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b5038-0ef3-48ff-809c-a132d116a4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:34:21.504095 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-427b5038-0ef3-48ff-809c-a132d116a4e1 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:34:21.505278 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-427b5038-0ef3-48ff-809c-a132d116a4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:34:21.513462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 749/1494] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:34:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:34:21.521456 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d48601-0d5a-42a3-b624-c652e38ab35e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:34:21.521921 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d48601-0d5a-42a3-b624-c652e38ab35e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:34:21.522026 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d48601-0d5a-42a3-b624-c652e38ab35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:34:21.540222 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-e0d48601-0d5a-42a3-b624-c652e38ab35e None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:34:21.541384 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e0d48601-0d5a-42a3-b624-c652e38ab35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:34:21.549063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 746/1495] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:34:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:34:21.553442 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b5038-0ef3-48ff-809c-a132d116a4e1 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:34:21.563007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b5038-0ef3-48ff-809c-a132d116a4e1 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:34:21.567447 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e373a-3453-4dcc-8f9b-ac218d1b6574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:34:21.568119 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e373a-3453-4dcc-8f9b-ac218d1b6574 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:34:21.568400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e373a-3453-4dcc-8f9b-ac218d1b6574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:34:21.568671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e373a-3453-4dcc-8f9b-ac218d1b6574 admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:34:21.569382 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e373a-3453-4dcc-8f9b-ac218d1b6574 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:34:21.569685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e373a-3453-4dcc-8f9b-ac218d1b6574 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:34:21.570690 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: WARNING py.warnings [None req-116e373a-3453-4dcc-8f9b-ac218d1b6574 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 06 10:34:21.570690 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: warnings.warn(msg) Jun 06 10:34:21.570690 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: Jun 06 10:34:21.571585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e373a-3453-4dcc-8f9b-ac218d1b6574 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:34:21.583425 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 750/1496] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:34:21.591899 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d48601-0d5a-42a3-b624-c652e38ab35e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:34:21.600931 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d48601-0d5a-42a3-b624-c652e38ab35e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:34:21.605428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff86d44-27f4-4fe7-8890-6ce2c6243ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:34:21.606064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff86d44-27f4-4fe7-8890-6ce2c6243ec0 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:34:21.606349 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff86d44-27f4-4fe7-8890-6ce2c6243ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:34:21.606628 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff86d44-27f4-4fe7-8890-6ce2c6243ec0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:34:21.607320 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff86d44-27f4-4fe7-8890-6ce2c6243ec0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:34:21.609085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: WARNING py.warnings [None req-1ff86d44-27f4-4fe7-8890-6ce2c6243ec0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 10:34:21.609085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: warnings.warn(msg) Jun 06 10:34:21.609085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: Jun 06 10:34:21.610262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff86d44-27f4-4fe7-8890-6ce2c6243ec0 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:34:21.649414 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 747/1497] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:34:21 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:34:21.653481 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e373a-3453-4dcc-8f9b-ac218d1b6574 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:34:21.663344 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e373a-3453-4dcc-8f9b-ac218d1b6574 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:34:21.718560 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccabfe-c3c6-4ae3-ae5a-3462774c5a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:34:21.719214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccabfe-c3c6-4ae3-ae5a-3462774c5a09 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:34:21.719497 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccabfe-c3c6-4ae3-ae5a-3462774c5a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:34:21.719771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccabfe-c3c6-4ae3-ae5a-3462774c5a09 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:34:21.720482 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adccabfe-c3c6-4ae3-ae5a-3462774c5a09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:34:21.721494 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: WARNING py.warnings [None req-adccabfe-c3c6-4ae3-ae5a-3462774c5a09 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 10:34:21.721494 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: warnings.warn(msg) Jun 06 10:34:21.721494 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: Jun 06 10:34:21.722679 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adccabfe-c3c6-4ae3-ae5a-3462774c5a09 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:34:21.741623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 751/1498] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:34:21 2022] POST /identity/v3/users => generated 290 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:34:21.745703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff86d44-27f4-4fe7-8890-6ce2c6243ec0 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:34:21.756432 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff86d44-27f4-4fe7-8890-6ce2c6243ec0 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:34:21.760817 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ed1b1-b7fa-4da5-91e9-ca480d6c8e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:34:21.761459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ed1b1-b7fa-4da5-91e9-ca480d6c8e48 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:34:21.761738 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ed1b1-b7fa-4da5-91e9-ca480d6c8e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:34:21.762011 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ed1b1-b7fa-4da5-91e9-ca480d6c8e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:34:21.763532 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ed1b1-b7fa-4da5-91e9-ca480d6c8e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:34:21.764607 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: WARNING py.warnings [None req-b06ed1b1-b7fa-4da5-91e9-ca480d6c8e48 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 06 10:34:21.764607 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: warnings.warn(msg) Jun 06 10:34:21.764607 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: Jun 06 10:34:21.765479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ed1b1-b7fa-4da5-91e9-ca480d6c8e48 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:34:21.779001 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 748/1499] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:34:21.783288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccabfe-c3c6-4ae3-ae5a-3462774c5a09 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:34:21.793138 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccabfe-c3c6-4ae3-ae5a-3462774c5a09 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:34:21.798286 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d356c-b475-4f77-9bb7-daa8b3d10e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:34:21.798939 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d356c-b475-4f77-9bb7-daa8b3d10e0e admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:34:21.799213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d356c-b475-4f77-9bb7-daa8b3d10e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:34:21.799484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d356c-b475-4f77-9bb7-daa8b3d10e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:34:21.800234 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3d356c-b475-4f77-9bb7-daa8b3d10e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:34:21.801238 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: WARNING py.warnings [None req-3e3d356c-b475-4f77-9bb7-daa8b3d10e0e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 06 10:34:21.801238 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: warnings.warn(msg) Jun 06 10:34:21.801238 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: Jun 06 10:34:21.802171 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3d356c-b475-4f77-9bb7-daa8b3d10e0e admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:34:21.815161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 752/1500] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:34:21.819178 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ed1b1-b7fa-4da5-91e9-ca480d6c8e48 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:34:21.829005 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ed1b1-b7fa-4da5-91e9-ca480d6c8e48 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:34:21.833427 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6aea9-af24-4814-97cd-311260f04542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:34:21.834294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6aea9-af24-4814-97cd-311260f04542 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:34:21.834573 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6aea9-af24-4814-97cd-311260f04542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:34:21.834848 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6aea9-af24-4814-97cd-311260f04542 admin admin] PATH_INFO: `/v3/projects/d14b4e83ddc747de99659aedc5212279/users/874f0c670d034e839aa2a4edd4b5324f/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:34:21.850299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a6aea9-af24-4814-97cd-311260f04542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d14b4e83ddc747de99659aedc5212279, user_id=874f0c670d034e839aa2a4edd4b5324f, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:34:21.851327 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: WARNING py.warnings [None req-f9a6aea9-af24-4814-97cd-311260f04542 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 10:34:21.851327 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: warnings.warn(msg) Jun 06 10:34:21.851327 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: Jun 06 10:34:21.853265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a6aea9-af24-4814-97cd-311260f04542 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:34:21.866872 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 749/1501] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:34:21 2022] PUT /identity/v3/projects/d14b4e83ddc747de99659aedc5212279/users/874f0c670d034e839aa2a4edd4b5324f/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:34:21.879401 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d529ea4-5c5d-4377-bbd8-61858e64087a None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:34:21.879691 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d529ea4-5c5d-4377-bbd8-61858e64087a None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:34:21.880007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d529ea4-5c5d-4377-bbd8-61858e64087a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:34:21.906269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-2d529ea4-5c5d-4377-bbd8-61858e64087a None None] MFA Rules not processed for user `874f0c670d034e839aa2a4edd4b5324f`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:34:21.907395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-2d529ea4-5c5d-4377-bbd8-61858e64087a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:34:21.969169 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 753/1502] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:34:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:34:21.982395 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3644d5-a6bc-472d-9539-e11f0453a966 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:34:21.982681 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3644d5-a6bc-472d-9539-e11f0453a966 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:34:21.982958 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3644d5-a6bc-472d-9539-e11f0453a966 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:34:21.984326 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 750/1503] 10.222.0.42 () {54 vars in 963 bytes} [Mon Jun 6 10:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:34:21.989144 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02207cd4-8224-494e-ba5a-be8cfb35e0e1 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:34:21.989434 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02207cd4-8224-494e-ba5a-be8cfb35e0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:34:21.989715 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02207cd4-8224-494e-ba5a-be8cfb35e0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:34:22.006939 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-02207cd4-8224-494e-ba5a-be8cfb35e0e1 None None] MFA Rules not processed for user `2b8836193f1a47e1a62ea5167e01b26e`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:34:22.008153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-02207cd4-8224-494e-ba5a-be8cfb35e0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:34:22.052508 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:34:22.061405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 754/1504] 10.222.0.42 () {58 vars in 1060 bytes} [Mon Jun 6 10:34:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:34:22.069126 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3644d5-a6bc-472d-9539-e11f0453a966 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:34:22.079643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3644d5-a6bc-472d-9539-e11f0453a966 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:34:22.083186 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8b8a0-41c7-4eff-b843-ae7bbcb9c3be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:34:22.083836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8b8a0-41c7-4eff-b843-ae7bbcb9c3be service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:34:22.084123 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8b8a0-41c7-4eff-b843-ae7bbcb9c3be service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:34:22.084401 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8b8a0-41c7-4eff-b843-ae7bbcb9c3be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:34:22.092533 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8b8a0-41c7-4eff-b843-ae7bbcb9c3be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:34:22.093957 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8b8a0-41c7-4eff-b843-ae7bbcb9c3be service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:34:22.102521 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 751/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:34:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:34:22.144472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb00793-3cf9-442f-8768-7341345d11b6 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:34:22.144762 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb00793-3cf9-442f-8768-7341345d11b6 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:34:22.145040 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb00793-3cf9-442f-8768-7341345d11b6 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:34:22.146311 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 755/1506] 10.222.0.42 () {52 vars in 829 bytes} [Mon Jun 6 10:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:34:22.150168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8b8a0-41c7-4eff-b843-ae7bbcb9c3be service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:34:22.211365 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8b8a0-41c7-4eff-b843-ae7bbcb9c3be service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:34:22.216395 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55fee3-88f9-4bc2-b1b0-a0fac9d39022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:34:22.217261 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55fee3-88f9-4bc2-b1b0-a0fac9d39022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:34:22.217540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55fee3-88f9-4bc2-b1b0-a0fac9d39022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:34:22.217816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55fee3-88f9-4bc2-b1b0-a0fac9d39022 admin admin] PATH_INFO: `/v3/users/874f0c670d034e839aa2a4edd4b5324f` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:34:22.218987 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55fee3-88f9-4bc2-b1b0-a0fac9d39022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=874f0c670d034e839aa2a4edd4b5324f)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:34:22.220037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: WARNING py.warnings [None req-ce55fee3-88f9-4bc2-b1b0-a0fac9d39022 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 10:34:22.220037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: warnings.warn(msg) Jun 06 10:34:22.220037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: Jun 06 10:34:22.221220 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55fee3-88f9-4bc2-b1b0-a0fac9d39022 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:34:22.288034 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ce55fee3-88f9-4bc2-b1b0-a0fac9d39022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '874f0c670d034e839aa2a4edd4b5324f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:34:22.301923 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ce55fee3-88f9-4bc2-b1b0-a0fac9d39022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '874f0c670d034e839aa2a4edd4b5324f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:34:22.302584 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ce55fee3-88f9-4bc2-b1b0-a0fac9d39022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '874f0c670d034e839aa2a4edd4b5324f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:34:22.303304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ce55fee3-88f9-4bc2-b1b0-a0fac9d39022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '874f0c670d034e839aa2a4edd4b5324f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:34:22.321400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ce55fee3-88f9-4bc2-b1b0-a0fac9d39022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '874f0c670d034e839aa2a4edd4b5324f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:34:22.325318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:34:22.337537 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 752/1507] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:34:22 2022] DELETE /identity/v3/users/874f0c670d034e839aa2a4edd4b5324f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:34:22.342416 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804fa72-bb2c-42cb-8d65-d6515b90932b None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:34:22.342697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804fa72-bb2c-42cb-8d65-d6515b90932b None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:34:22.342970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804fa72-bb2c-42cb-8d65-d6515b90932b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:34:22.360516 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-5804fa72-bb2c-42cb-8d65-d6515b90932b None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:34:22.361693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-5804fa72-bb2c-42cb-8d65-d6515b90932b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:34:22.420684 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 756/1508] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:34:22.430120 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55fee3-88f9-4bc2-b1b0-a0fac9d39022 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:34:22.431510 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ce55fee3-88f9-4bc2-b1b0-a0fac9d39022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:34:22.489232 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55fee3-88f9-4bc2-b1b0-a0fac9d39022 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:34:22.492637 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431e24c-3af0-48e7-b0ff-0c6f3bbe6e10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:34:22.493272 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431e24c-3af0-48e7-b0ff-0c6f3bbe6e10 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:34:22.493550 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431e24c-3af0-48e7-b0ff-0c6f3bbe6e10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:34:22.493821 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431e24c-3af0-48e7-b0ff-0c6f3bbe6e10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:34:22.502817 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431e24c-3af0-48e7-b0ff-0c6f3bbe6e10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:34:22.504270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431e24c-3af0-48e7-b0ff-0c6f3bbe6e10 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:34:22.513770 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 753/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:34:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:34:22.841825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804fa72-bb2c-42cb-8d65-d6515b90932b None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:34:22.843305 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-5804fa72-bb2c-42cb-8d65-d6515b90932b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:34:22.857985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804fa72-bb2c-42cb-8d65-d6515b90932b None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:34:22.867619 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5bde0e-1e6d-4019-968c-0cf23c6b4eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:34:22.868531 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bde0e-1e6d-4019-968c-0cf23c6b4eb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:34:22.868821 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bde0e-1e6d-4019-968c-0cf23c6b4eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:34:22.869088 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bde0e-1e6d-4019-968c-0cf23c6b4eb4 admin admin] PATH_INFO: `/v3/projects/d14b4e83ddc747de99659aedc5212279` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:34:22.870273 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5bde0e-1e6d-4019-968c-0cf23c6b4eb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d14b4e83ddc747de99659aedc5212279)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:34:22.871298 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: WARNING py.warnings [None req-9b5bde0e-1e6d-4019-968c-0cf23c6b4eb4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 10:34:22.871298 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: warnings.warn(msg) Jun 06 10:34:22.871298 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: Jun 06 10:34:22.872910 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5bde0e-1e6d-4019-968c-0cf23c6b4eb4 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:34:22.934877 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9b5bde0e-1e6d-4019-968c-0cf23c6b4eb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd14b4e83ddc747de99659aedc5212279', 'request_id': 'req-9b5bde0e-1e6d-4019-968c-0cf23c6b4eb4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:34:22.940779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9b5bde0e-1e6d-4019-968c-0cf23c6b4eb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd14b4e83ddc747de99659aedc5212279', 'request_id': 'req-9b5bde0e-1e6d-4019-968c-0cf23c6b4eb4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:34:22.951869 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9b5bde0e-1e6d-4019-968c-0cf23c6b4eb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd14b4e83ddc747de99659aedc5212279', 'request_id': 'req-9b5bde0e-1e6d-4019-968c-0cf23c6b4eb4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:34:22.970402 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9b5bde0e-1e6d-4019-968c-0cf23c6b4eb4 admin admin] The token cache is being invalidate because project d14b4e83ddc747de99659aedc5212279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:34:22.970691 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9b5bde0e-1e6d-4019-968c-0cf23c6b4eb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:34:22.971319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9b5bde0e-1e6d-4019-968c-0cf23c6b4eb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:34:22.973009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 757/1510] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:34:22 2022] DELETE /identity/v3/projects/d14b4e83ddc747de99659aedc5212279 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:34:32.509337 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:34:32.967426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:35:59.411472 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e20f28-6f8f-40c4-b0f4-1b133f283dc0 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:35:59.412129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e20f28-6f8f-40c4-b0f4-1b133f283dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:35:59.412129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e20f28-6f8f-40c4-b0f4-1b133f283dc0 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:35:59.413836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 10:35:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:35:59.423817 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5557518-ca69-4bbf-b3fc-cb337bd4cf59 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:35:59.424352 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5557518-ca69-4bbf-b3fc-cb337bd4cf59 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:35:59.424518 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5557518-ca69-4bbf-b3fc-cb337bd4cf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:35:59.444236 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-c5557518-ca69-4bbf-b3fc-cb337bd4cf59 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:35:59.446226 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-c5557518-ca69-4bbf-b3fc-cb337bd4cf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:35:59.514580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 10:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:35:59.691129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e20f28-6f8f-40c4-b0f4-1b133f283dc0 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:35:59.692844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-84e20f28-6f8f-40c4-b0f4-1b133f283dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:35:59.766025 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e20f28-6f8f-40c4-b0f4-1b133f283dc0 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:35:59.772421 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f676c40-673b-4498-bf64-e3ac677d721a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:35:59.773071 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f676c40-673b-4498-bf64-e3ac677d721a service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:35:59.773357 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f676c40-673b-4498-bf64-e3ac677d721a service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:35:59.773631 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f676c40-673b-4498-bf64-e3ac677d721a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:35:59.782055 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f676c40-673b-4498-bf64-e3ac677d721a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:35:59.783664 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f676c40-673b-4498-bf64-e3ac677d721a service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:35:59.793143 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 755/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:35:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:36:00.206464 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d1d19-7c20-4888-9f64-4d9d671cf3b4 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:00.206755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d1d19-7c20-4888-9f64-4d9d671cf3b4 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:00.207064 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d1d19-7c20-4888-9f64-4d9d671cf3b4 None None] PATH_INFO: `/v3` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:00.208417 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 759/1514] 10.222.0.46 () {54 vars in 925 bytes} [Mon Jun 6 10:36:00 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:36:00.214439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a268d2-1c15-4458-9c0d-78fbb33a256b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:00.214735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a268d2-1c15-4458-9c0d-78fbb33a256b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:00.215017 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a268d2-1c15-4458-9c0d-78fbb33a256b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:00.232730 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-46a268d2-1c15-4458-9c0d-78fbb33a256b None None] MFA Rules not processed for user `361be69f1c0a41c18afd206bccf56447`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:36:00.233965 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-46a268d2-1c15-4458-9c0d-78fbb33a256b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:36:00.297785 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 756/1515] 10.222.0.46 () {58 vars in 1015 bytes} [Mon Jun 6 10:36:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:36:00.672956 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8202d-39c5-470a-ad2e-9522d0dd426a None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:00.673483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8202d-39c5-470a-ad2e-9522d0dd426a None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:00.673483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8202d-39c5-470a-ad2e-9522d0dd426a None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:00.675470 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 760/1516] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 10:36:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:36:00.680257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67c3d7-8de9-4a63-b361-dc7c729b8de2 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:00.680682 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67c3d7-8de9-4a63-b361-dc7c729b8de2 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:00.680796 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67c3d7-8de9-4a63-b361-dc7c729b8de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:00.698618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-bd67c3d7-8de9-4a63-b361-dc7c729b8de2 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:36:00.699819 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bd67c3d7-8de9-4a63-b361-dc7c729b8de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:36:00.707665 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 757/1517] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 10:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:36:00.859929 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8202d-39c5-470a-ad2e-9522d0dd426a None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:36:00.874790 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8202d-39c5-470a-ad2e-9522d0dd426a None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:36:00.880810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ef89f-6819-4f43-aeb8-d4a58cef2fef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:36:00.881442 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ef89f-6819-4f43-aeb8-d4a58cef2fef service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:00.881728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ef89f-6819-4f43-aeb8-d4a58cef2fef service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:00.882008 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ef89f-6819-4f43-aeb8-d4a58cef2fef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:00.890106 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ef89f-6819-4f43-aeb8-d4a58cef2fef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:36:00.891684 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ef89f-6819-4f43-aeb8-d4a58cef2fef service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:36:00.901580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 761/1518] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:36:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:36:04.225869 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd254d1-5d65-4280-9a5d-bccdda1a350e None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:04.226437 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd254d1-5d65-4280-9a5d-bccdda1a350e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:04.226437 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd254d1-5d65-4280-9a5d-bccdda1a350e None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:04.227772 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 758/1519] 10.222.0.42 () {54 vars in 960 bytes} [Mon Jun 6 10:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:36:04.232732 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c3dde-26cc-4d9b-bbe9-528107d9b37c None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:04.233130 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c3dde-26cc-4d9b-bbe9-528107d9b37c None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:04.233288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c3dde-26cc-4d9b-bbe9-528107d9b37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:04.251872 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-0f1c3dde-26cc-4d9b-bbe9-528107d9b37c None None] MFA Rules not processed for user `361be69f1c0a41c18afd206bccf56447`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:36:04.253079 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-0f1c3dde-26cc-4d9b-bbe9-528107d9b37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:36:04.261516 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 762/1520] 10.222.0.42 () {58 vars in 1057 bytes} [Mon Jun 6 10:36:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:36:04.268890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd254d1-5d65-4280-9a5d-bccdda1a350e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:36:04.282315 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd254d1-5d65-4280-9a5d-bccdda1a350e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:36:04.287760 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d46c6-0bee-4c0d-aa52-351f06b6a781 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361be69f1c0a41c18afd206bccf56447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:36:04.288403 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d46c6-0bee-4c0d-aa52-351f06b6a781 service placement] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:04.288686 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d46c6-0bee-4c0d-aa52-351f06b6a781 service placement] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:04.288958 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d46c6-0bee-4c0d-aa52-351f06b6a781 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:04.297608 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d46c6-0bee-4c0d-aa52-351f06b6a781 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:36:04.299927 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d46c6-0bee-4c0d-aa52-351f06b6a781 service placement] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:36:04.309598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 759/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:36:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:36:12.165779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4c9e8-7ca6-457d-9357-9747dac423b0 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:12.166323 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4c9e8-7ca6-457d-9357-9747dac423b0 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:12.166459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4c9e8-7ca6-457d-9357-9747dac423b0 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:12.167763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 10:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:36:12.172729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568b80d-b477-4c46-b48a-27fdd1d72531 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:12.173157 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568b80d-b477-4c46-b48a-27fdd1d72531 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:12.173283 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568b80d-b477-4c46-b48a-27fdd1d72531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:12.191505 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-e568b80d-b477-4c46-b48a-27fdd1d72531 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:36:12.192733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e568b80d-b477-4c46-b48a-27fdd1d72531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:36:12.200435 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 10:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:36:12.356414 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4c9e8-7ca6-457d-9357-9747dac423b0 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:36:12.371288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4c9e8-7ca6-457d-9357-9747dac423b0 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:36:12.376737 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f1c0a-c428-4ca4-8d3c-fa6a5b821769 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:36:12.378454 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f1c0a-c428-4ca4-8d3c-fa6a5b821769 service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:12.378751 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f1c0a-c428-4ca4-8d3c-fa6a5b821769 service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:12.379026 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f1c0a-c428-4ca4-8d3c-fa6a5b821769 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:12.387488 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f1c0a-c428-4ca4-8d3c-fa6a5b821769 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:36:12.389092 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f1c0a-c428-4ca4-8d3c-fa6a5b821769 service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:36:12.399765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 764/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:36:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:36:13.270897 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9632f5-2448-43fb-bbcb-bac6c071b2d5 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:13.271420 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9632f5-2448-43fb-bbcb-bac6c071b2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:13.271420 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9632f5-2448-43fb-bbcb-bac6c071b2d5 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:13.273000 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 10:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:36:13.277843 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca6cfb-244d-4d8f-a492-1ef33f9fa65e None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:13.278272 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca6cfb-244d-4d8f-a492-1ef33f9fa65e None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:13.278402 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca6cfb-244d-4d8f-a492-1ef33f9fa65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:13.296184 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-80ca6cfb-244d-4d8f-a492-1ef33f9fa65e None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:36:13.297376 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-80ca6cfb-244d-4d8f-a492-1ef33f9fa65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:36:13.305581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 10:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:36:13.456878 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9632f5-2448-43fb-bbcb-bac6c071b2d5 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:36:13.471087 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9632f5-2448-43fb-bbcb-bac6c071b2d5 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:36:13.476611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ca588-ce41-431f-a665-5f939340e421 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:36:13.477241 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ca588-ce41-431f-a665-5f939340e421 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:13.477534 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ca588-ce41-431f-a665-5f939340e421 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:13.477813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ca588-ce41-431f-a665-5f939340e421 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:13.485866 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ca588-ce41-431f-a665-5f939340e421 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:36:13.487963 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ca588-ce41-431f-a665-5f939340e421 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:36:13.497241 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 762/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:36:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:36:17.716733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc92de-0c83-42d0-a722-f263e45a772f None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:17.717288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc92de-0c83-42d0-a722-f263e45a772f None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:17.717288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc92de-0c83-42d0-a722-f263e45a772f None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:17.718645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 10:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:36:17.724051 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f740f1-eb5c-42b7-be28-4dfdf735fecd None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:17.724456 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f740f1-eb5c-42b7-be28-4dfdf735fecd None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:17.724610 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f740f1-eb5c-42b7-be28-4dfdf735fecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:17.742649 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-43f740f1-eb5c-42b7-be28-4dfdf735fecd None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:36:17.743868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-43f740f1-eb5c-42b7-be28-4dfdf735fecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:36:17.751539 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 10:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:36:17.791768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc92de-0c83-42d0-a722-f263e45a772f None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:36:17.793456 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-04bc92de-0c83-42d0-a722-f263e45a772f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:36:17.849469 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc92de-0c83-42d0-a722-f263e45a772f None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:36:17.853185 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33bb6e-1890-4bc9-bbd0-883c331de2c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:36:17.853820 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33bb6e-1890-4bc9-bbd0-883c331de2c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:17.854101 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33bb6e-1890-4bc9-bbd0-883c331de2c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:17.854372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33bb6e-1890-4bc9-bbd0-883c331de2c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:17.863198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33bb6e-1890-4bc9-bbd0-883c331de2c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:36:17.864639 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33bb6e-1890-4bc9-bbd0-883c331de2c5 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:36:17.873936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 767/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:36:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:36:20.118338 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98bf83e-12a0-4bd8-a4d0-81d7fc066f98 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:20.118908 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98bf83e-12a0-4bd8-a4d0-81d7fc066f98 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:20.118908 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98bf83e-12a0-4bd8-a4d0-81d7fc066f98 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:20.120316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 10:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:36:20.125407 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f00ae4-c310-4bee-afe3-5b3c0451a2ec None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:20.125820 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f00ae4-c310-4bee-afe3-5b3c0451a2ec None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:20.125929 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f00ae4-c310-4bee-afe3-5b3c0451a2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:20.146728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-72f00ae4-c310-4bee-afe3-5b3c0451a2ec None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:36:20.148303 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-72f00ae4-c310-4bee-afe3-5b3c0451a2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:36:20.156064 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 10:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:36:20.192678 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98bf83e-12a0-4bd8-a4d0-81d7fc066f98 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:36:20.204499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98bf83e-12a0-4bd8-a4d0-81d7fc066f98 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:36:20.208680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6989b8-d879-405b-b47d-4d782daf81b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:36:20.209311 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6989b8-d879-405b-b47d-4d782daf81b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:20.209592 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6989b8-d879-405b-b47d-4d782daf81b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:20.209861 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6989b8-d879-405b-b47d-4d782daf81b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:20.218762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6989b8-d879-405b-b47d-4d782daf81b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:36:20.220214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6989b8-d879-405b-b47d-4d782daf81b2 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:36:20.229400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 765/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:36:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:36:24.062574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3b69f-a0b4-4114-bbc3-356cb801e004 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:24.063189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3b69f-a0b4-4114-bbc3-356cb801e004 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:24.063189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3b69f-a0b4-4114-bbc3-356cb801e004 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:24.065151 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 10:36:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:36:24.070352 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c6f6e-9af3-4d68-9d85-b5c6a4f08ac1 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:24.070795 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c6f6e-9af3-4d68-9d85-b5c6a4f08ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:24.070904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c6f6e-9af3-4d68-9d85-b5c6a4f08ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:24.077176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=91 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:36:24.089731 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-e12c6f6e-9af3-4d68-9d85-b5c6a4f08ac1 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:36:24.090932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e12c6f6e-9af3-4d68-9d85-b5c6a4f08ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:36:24.098717 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 10:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:36:24.386157 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3b69f-a0b4-4114-bbc3-356cb801e004 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:36:24.387616 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-6ee3b69f-a0b4-4114-bbc3-356cb801e004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:36:24.393368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=75 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:36:24.445685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3b69f-a0b4-4114-bbc3-356cb801e004 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:36:24.449389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c6cb3-9a48-47b7-aca8-3e848f600ab5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:36:24.450026 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c6cb3-9a48-47b7-aca8-3e848f600ab5 service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:24.450305 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c6cb3-9a48-47b7-aca8-3e848f600ab5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:24.450575 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c6cb3-9a48-47b7-aca8-3e848f600ab5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:24.459580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c6cb3-9a48-47b7-aca8-3e848f600ab5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:36:24.461067 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c6cb3-9a48-47b7-aca8-3e848f600ab5 service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:36:24.470279 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 770/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:36:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:36:26.776334 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacdd13-6c57-466b-be74-065365627ebe None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:26.776871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacdd13-6c57-466b-be74-065365627ebe None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:26.776871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacdd13-6c57-466b-be74-065365627ebe None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:26.778243 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 10:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:36:26.783370 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13bbce-ba2e-4b4f-b817-3e16570af231 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:26.783838 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13bbce-ba2e-4b4f-b817-3e16570af231 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:26.783952 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13bbce-ba2e-4b4f-b817-3e16570af231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:26.802763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-9b13bbce-ba2e-4b4f-b817-3e16570af231 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:36:26.804126 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-9b13bbce-ba2e-4b4f-b817-3e16570af231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:36:26.811563 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 10:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:36:27.089122 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacdd13-6c57-466b-be74-065365627ebe None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:36:27.090788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1eacdd13-6c57-466b-be74-065365627ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:36:27.104275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacdd13-6c57-466b-be74-065365627ebe None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:36:27.107731 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9912230-6f9e-4c95-ae95-e5bac457ed2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:36:27.108380 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9912230-6f9e-4c95-ae95-e5bac457ed2f service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:36:27.108659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9912230-6f9e-4c95-ae95-e5bac457ed2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:36:27.108929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9912230-6f9e-4c95-ae95-e5bac457ed2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:36:27.117728 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9912230-6f9e-4c95-ae95-e5bac457ed2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:36:27.119136 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9912230-6f9e-4c95-ae95-e5bac457ed2f service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:36:27.128272 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 768/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:36:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:36:36.797460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=28 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:36:37.124296 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=8 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:37:58.107693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76da40b-d599-4b21-b8a7-be3baa58a82a None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.108763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76da40b-d599-4b21-b8a7-be3baa58a82a None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.109212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76da40b-d599-4b21-b8a7-be3baa58a82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.138182 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-e76da40b-d599-4b21-b8a7-be3baa58a82a None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:37:58.139671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-e76da40b-d599-4b21-b8a7-be3baa58a82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:37:58.154599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 772/1540] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:37:58.164485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9b77e-f536-4441-8311-c25253f87351 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.165763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9b77e-f536-4441-8311-c25253f87351 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.166224 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9b77e-f536-4441-8311-c25253f87351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.185365 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de16f7-babe-40a2-ac1f-14f01bba8742 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.186239 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de16f7-babe-40a2-ac1f-14f01bba8742 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.186682 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de16f7-babe-40a2-ac1f-14f01bba8742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.200070 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ddc9b77e-f536-4441-8311-c25253f87351 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:37:58.202458 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ddc9b77e-f536-4441-8311-c25253f87351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:37:58.220559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-22de16f7-babe-40a2-ac1f-14f01bba8742 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:37:58.222065 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-22de16f7-babe-40a2-ac1f-14f01bba8742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:37:58.224870 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 769/1541] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:37:58.226344 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3982772-1e59-42e1-96aa-2d3f35940408 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.228012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3982772-1e59-42e1-96aa-2d3f35940408 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.228841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3982772-1e59-42e1-96aa-2d3f35940408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.241254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 773/1542] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:37:58.242718 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de16f7-babe-40a2-ac1f-14f01bba8742 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:58.265913 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de16f7-babe-40a2-ac1f-14f01bba8742 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:58.267764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b3982772-1e59-42e1-96aa-2d3f35940408 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:37:58.269986 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b3982772-1e59-42e1-96aa-2d3f35940408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:37:58.283991 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3ca25-b036-420d-97d1-ee35f85f9c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:58.284933 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3ca25-b036-420d-97d1-ee35f85f9c6c admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.285384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3ca25-b036-420d-97d1-ee35f85f9c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.285832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3ca25-b036-420d-97d1-ee35f85f9c6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.286728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3ca25-b036-420d-97d1-ee35f85f9c6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:37:58.287137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 770/1543] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:37:58.287370 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3ca25-b036-420d-97d1-ee35f85f9c6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:58.288448 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba16722-e1a2-447a-8bf8-bd66b2fa68a8 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.288733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba16722-e1a2-447a-8bf8-bd66b2fa68a8 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.289009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba16722-e1a2-447a-8bf8-bd66b2fa68a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.289744 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3ca25-b036-420d-97d1-ee35f85f9c6c admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:58.323590 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 774/1544] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:37:58.324998 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-0ba16722-e1a2-447a-8bf8-bd66b2fa68a8 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:37:58.326221 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0ba16722-e1a2-447a-8bf8-bd66b2fa68a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:37:58.328126 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9504b8-6081-4125-93d2-35e86961a9be None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.328413 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9504b8-6081-4125-93d2-35e86961a9be None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.328689 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9504b8-6081-4125-93d2-35e86961a9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.340670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 771/1545] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:37:58.342112 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa67ec-eeb0-47fc-9a1f-1a3353738571 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.342412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa67ec-eeb0-47fc-9a1f-1a3353738571 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.342684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa67ec-eeb0-47fc-9a1f-1a3353738571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.350530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-eb9504b8-6081-4125-93d2-35e86961a9be None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:37:58.351970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-eb9504b8-6081-4125-93d2-35e86961a9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:37:58.361455 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 775/1546] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:37:58.362638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc220eec-7df1-4e25-92cc-8ce9002e0d9f None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.362922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc220eec-7df1-4e25-92cc-8ce9002e0d9f None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.363199 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc220eec-7df1-4e25-92cc-8ce9002e0d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.368793 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-8eaa67ec-eeb0-47fc-9a1f-1a3353738571 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:37:58.369984 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-8eaa67ec-eeb0-47fc-9a1f-1a3353738571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:37:58.380898 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 772/1547] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:37:58.381706 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa67ec-eeb0-47fc-9a1f-1a3353738571 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:58.397008 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-dc220eec-7df1-4e25-92cc-8ce9002e0d9f None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:37:58.398299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa67ec-eeb0-47fc-9a1f-1a3353738571 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:58.398444 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-dc220eec-7df1-4e25-92cc-8ce9002e0d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:37:58.403244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861dff45-d365-4353-85b1-18acca2f103d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:58.403949 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dff45-d365-4353-85b1-18acca2f103d admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.404226 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dff45-d365-4353-85b1-18acca2f103d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.404494 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dff45-d365-4353-85b1-18acca2f103d admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.405389 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861dff45-d365-4353-85b1-18acca2f103d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:58.407651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 776/1548] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:37:58.409064 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65068d4b-1571-43dd-9403-4f6cda2529e2 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.409194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861dff45-d365-4353-85b1-18acca2f103d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:58.409383 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65068d4b-1571-43dd-9403-4f6cda2529e2 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.409627 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65068d4b-1571-43dd-9403-4f6cda2529e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.438047 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-65068d4b-1571-43dd-9403-4f6cda2529e2 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:37:58.439569 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-65068d4b-1571-43dd-9403-4f6cda2529e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:37:58.449159 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 777/1549] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:37:58.454797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65068d4b-1571-43dd-9403-4f6cda2529e2 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:58.459638 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 773/1550] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/projects => generated 375 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:37:58.461064 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65068d4b-1571-43dd-9403-4f6cda2529e2 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:58.461366 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70835f2f-c20d-4943-a62e-cfc21d73b3fd None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.461932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70835f2f-c20d-4943-a62e-cfc21d73b3fd None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.462366 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70835f2f-c20d-4943-a62e-cfc21d73b3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.486826 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-70835f2f-c20d-4943-a62e-cfc21d73b3fd None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:37:58.488046 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-70835f2f-c20d-4943-a62e-cfc21d73b3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:37:58.523636 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1032803a-40fc-40c7-b1b4-7010c9df0aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:58.524337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032803a-40fc-40c7-b1b4-7010c9df0aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.524713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032803a-40fc-40c7-b1b4-7010c9df0aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.524995 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032803a-40fc-40c7-b1b4-7010c9df0aa2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.525717 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1032803a-40fc-40c7-b1b4-7010c9df0aa2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:37:58.526026 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1032803a-40fc-40c7-b1b4-7010c9df0aa2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:58.527610 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1032803a-40fc-40c7-b1b4-7010c9df0aa2 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:58.541233 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 778/1551] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:37:58.542251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1032803a-40fc-40c7-b1b4-7010c9df0aa2 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:58.549483 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 774/1552] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:37:58.550426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67608ab-c91f-44b2-8778-e855b8344dcf None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.550713 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67608ab-c91f-44b2-8778-e855b8344dcf None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.550989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67608ab-c91f-44b2-8778-e855b8344dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.556083 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1032803a-40fc-40c7-b1b4-7010c9df0aa2 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:58.560615 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e47a05-78d4-4fc9-a703-283939470571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:58.561264 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e47a05-78d4-4fc9-a703-283939470571 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.561551 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e47a05-78d4-4fc9-a703-283939470571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.561831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e47a05-78d4-4fc9-a703-283939470571 admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.563626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e47a05-78d4-4fc9-a703-283939470571 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:37:58.563806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e47a05-78d4-4fc9-a703-283939470571 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:58.566466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e47a05-78d4-4fc9-a703-283939470571 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:58.569213 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-e67608ab-c91f-44b2-8778-e855b8344dcf None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:37:58.570402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e67608ab-c91f-44b2-8778-e855b8344dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:37:58.578957 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 775/1553] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:37:58.579725 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67608ab-c91f-44b2-8778-e855b8344dcf None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:58.580141 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 779/1554] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:37:58.581391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288ed4b-5e54-44fd-ad13-743abed229d4 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.581679 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288ed4b-5e54-44fd-ad13-743abed229d4 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.581953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288ed4b-5e54-44fd-ad13-743abed229d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.589835 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67608ab-c91f-44b2-8778-e855b8344dcf None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:58.594584 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb230d7-a38c-460f-b585-1dd39acc9c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:58.595238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb230d7-a38c-460f-b585-1dd39acc9c4c admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.595516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb230d7-a38c-460f-b585-1dd39acc9c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.595802 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb230d7-a38c-460f-b585-1dd39acc9c4c admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.596522 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb230d7-a38c-460f-b585-1dd39acc9c4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:58.597530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: WARNING py.warnings [None req-fdb230d7-a38c-460f-b585-1dd39acc9c4c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 10:37:58.597530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: warnings.warn(msg) Jun 06 10:37:58.597530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: Jun 06 10:37:58.598711 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb230d7-a38c-460f-b585-1dd39acc9c4c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:58.600278 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-9288ed4b-5e54-44fd-ad13-743abed229d4 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:37:58.601479 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-9288ed4b-5e54-44fd-ad13-743abed229d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:37:58.611425 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 780/1555] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:37:58.613202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589e0eb-a26f-45b4-9c5b-70ba64f77725 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.613428 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589e0eb-a26f-45b4-9c5b-70ba64f77725 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.613705 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589e0eb-a26f-45b4-9c5b-70ba64f77725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.623040 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 776/1556] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/users => generated 288 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:37:58.623040 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92315a05-2295-4719-9409-6b152a5932ec None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.623040 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92315a05-2295-4719-9409-6b152a5932ec None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.623040 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92315a05-2295-4719-9409-6b152a5932ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.632853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-9589e0eb-a26f-45b4-9c5b-70ba64f77725 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:37:58.634053 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-9589e0eb-a26f-45b4-9c5b-70ba64f77725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:37:58.640603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-92315a05-2295-4719-9409-6b152a5932ec None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:37:58.641758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-92315a05-2295-4719-9409-6b152a5932ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:37:58.642084 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 781/1557] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:37:58.642832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589e0eb-a26f-45b4-9c5b-70ba64f77725 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:58.650530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 777/1558] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:37:58.651302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92315a05-2295-4719-9409-6b152a5932ec None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:58.652413 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589e0eb-a26f-45b4-9c5b-70ba64f77725 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:58.657276 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82527868-04b5-49e8-80b7-68324acf28be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:58.658013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82527868-04b5-49e8-80b7-68324acf28be admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.658305 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82527868-04b5-49e8-80b7-68324acf28be admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.658586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82527868-04b5-49e8-80b7-68324acf28be admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.659272 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82527868-04b5-49e8-80b7-68324acf28be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:58.660322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: WARNING py.warnings [None req-82527868-04b5-49e8-80b7-68324acf28be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 10:37:58.660322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: warnings.warn(msg) Jun 06 10:37:58.660322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: Jun 06 10:37:58.660973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92315a05-2295-4719-9409-6b152a5932ec None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:58.661499 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82527868-04b5-49e8-80b7-68324acf28be admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:58.665536 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1930d09-dba9-4969-9fc5-aa1042a707b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:58.666179 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1930d09-dba9-4969-9fc5-aa1042a707b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.666465 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1930d09-dba9-4969-9fc5-aa1042a707b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.666744 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1930d09-dba9-4969-9fc5-aa1042a707b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.667468 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1930d09-dba9-4969-9fc5-aa1042a707b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:37:58.667769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1930d09-dba9-4969-9fc5-aa1042a707b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:58.668891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: WARNING py.warnings [None req-b1930d09-dba9-4969-9fc5-aa1042a707b6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 06 10:37:58.668891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: warnings.warn(msg) Jun 06 10:37:58.668891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: Jun 06 10:37:58.669766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1930d09-dba9-4969-9fc5-aa1042a707b6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:58.683867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 778/1559] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:37:58.684746 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1930d09-dba9-4969-9fc5-aa1042a707b6 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:58.695661 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1930d09-dba9-4969-9fc5-aa1042a707b6 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:58.698493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 782/1560] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/projects => generated 403 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:37:58.699313 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82527868-04b5-49e8-80b7-68324acf28be admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:58.702083 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009d74a-cbc4-4363-a170-dece6bdb011a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:58.702754 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009d74a-cbc4-4363-a170-dece6bdb011a admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.703142 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009d74a-cbc4-4363-a170-dece6bdb011a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.703584 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009d74a-cbc4-4363-a170-dece6bdb011a admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.704422 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009d74a-cbc4-4363-a170-dece6bdb011a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:37:58.704738 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009d74a-cbc4-4363-a170-dece6bdb011a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:58.707671 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009d74a-cbc4-4363-a170-dece6bdb011a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:58.720014 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 779/1561] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:37:58.721367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe074904-fa34-4191-a579-43ba36dc8f3d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.721663 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe074904-fa34-4191-a579-43ba36dc8f3d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.721941 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe074904-fa34-4191-a579-43ba36dc8f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.740542 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-fe074904-fa34-4191-a579-43ba36dc8f3d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:37:58.741766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fe074904-fa34-4191-a579-43ba36dc8f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:37:58.771875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82527868-04b5-49e8-80b7-68324acf28be admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:58.776469 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562656af-d708-49d3-9670-a238bcc852ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:58.777125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562656af-d708-49d3-9670-a238bcc852ea admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.777410 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562656af-d708-49d3-9670-a238bcc852ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.777685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562656af-d708-49d3-9670-a238bcc852ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.778366 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562656af-d708-49d3-9670-a238bcc852ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:58.780297 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562656af-d708-49d3-9670-a238bcc852ea admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:58.784763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:37:58.791269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:37:58.806354 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 780/1562] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:37:58.807253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe074904-fa34-4191-a579-43ba36dc8f3d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:58.812439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 783/1563] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:37:58.812439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562656af-d708-49d3-9670-a238bcc852ea admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:58.816805 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe074904-fa34-4191-a579-43ba36dc8f3d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:58.873843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1d9ed-2352-4994-963c-171a62a9a2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:58.874676 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562656af-d708-49d3-9670-a238bcc852ea admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:58.874860 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1d9ed-2352-4994-963c-171a62a9a2df admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.875161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1d9ed-2352-4994-963c-171a62a9a2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.875161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1d9ed-2352-4994-963c-171a62a9a2df admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.876152 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1d9ed-2352-4994-963c-171a62a9a2df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:58.877729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1d9ed-2352-4994-963c-171a62a9a2df admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:58.879517 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc5e8c-39f0-496e-ae1a-ac93a857d762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:58.880205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc5e8c-39f0-496e-ae1a-ac93a857d762 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.880491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc5e8c-39f0-496e-ae1a-ac93a857d762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.880772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc5e8c-39f0-496e-ae1a-ac93a857d762 admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.881490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc5e8c-39f0-496e-ae1a-ac93a857d762 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:37:58.881800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc5e8c-39f0-496e-ae1a-ac93a857d762 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:58.883386 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc5e8c-39f0-496e-ae1a-ac93a857d762 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:58.890646 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 781/1564] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:37:58.890646 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b9fbe-b6ca-42a9-82df-f8cb45e11512 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.890646 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b9fbe-b6ca-42a9-82df-f8cb45e11512 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.890646 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b9fbe-b6ca-42a9-82df-f8cb45e11512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.899818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 784/1565] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:37:58.900628 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc5e8c-39f0-496e-ae1a-ac93a857d762 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:58.911416 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc5e8c-39f0-496e-ae1a-ac93a857d762 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:58.913119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-843b9fbe-b6ca-42a9-82df-f8cb45e11512 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:37:58.914322 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-843b9fbe-b6ca-42a9-82df-f8cb45e11512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:37:58.915905 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3970b1c-d114-4600-bd85-9a8e5393ac4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:58.916536 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3970b1c-d114-4600-bd85-9a8e5393ac4a admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.916814 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3970b1c-d114-4600-bd85-9a8e5393ac4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.917082 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3970b1c-d114-4600-bd85-9a8e5393ac4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.917756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3970b1c-d114-4600-bd85-9a8e5393ac4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:58.919622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3970b1c-d114-4600-bd85-9a8e5393ac4a admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:58.922816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 782/1566] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:37:58.923687 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b9fbe-b6ca-42a9-82df-f8cb45e11512 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:58.933125 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b9fbe-b6ca-42a9-82df-f8cb45e11512 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:58.938330 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f3275-7e9e-46af-a905-c30ff5f05d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:58.938973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f3275-7e9e-46af-a905-c30ff5f05d24 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:58.939269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f3275-7e9e-46af-a905-c30ff5f05d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:58.939531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f3275-7e9e-46af-a905-c30ff5f05d24 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:58.940497 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f3275-7e9e-46af-a905-c30ff5f05d24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:58.943012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f3275-7e9e-46af-a905-c30ff5f05d24 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:58.951169 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 785/1567] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:37:58.952092 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3970b1c-d114-4600-bd85-9a8e5393ac4a admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:58.964797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 783/1568] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/users => generated 302 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:37:58.965674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f3275-7e9e-46af-a905-c30ff5f05d24 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:58.975439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f3275-7e9e-46af-a905-c30ff5f05d24 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.020550 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3970b1c-d114-4600-bd85-9a8e5393ac4a admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.025632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2fa50-deac-4e37-add9-bb6cd06d7c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.026317 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2fa50-deac-4e37-add9-bb6cd06d7c08 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.026617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2fa50-deac-4e37-add9-bb6cd06d7c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.026887 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2fa50-deac-4e37-add9-bb6cd06d7c08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.027568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2fa50-deac-4e37-add9-bb6cd06d7c08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.029491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2fa50-deac-4e37-add9-bb6cd06d7c08 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.032996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef7950-43c8-4766-bb64-5522d0349eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.033659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef7950-43c8-4766-bb64-5522d0349eaa admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.033946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef7950-43c8-4766-bb64-5522d0349eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.034222 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef7950-43c8-4766-bb64-5522d0349eaa admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.034945 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef7950-43c8-4766-bb64-5522d0349eaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:37:59.035249 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef7950-43c8-4766-bb64-5522d0349eaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.036903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef7950-43c8-4766-bb64-5522d0349eaa admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.051367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 784/1569] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:37:59.052446 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef7950-43c8-4766-bb64-5522d0349eaa admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.064594 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 786/1570] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:37:58 2022] POST /identity/v3/projects => generated 401 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:37:59.064594 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2fa50-deac-4e37-add9-bb6cd06d7c08 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.065049 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef7950-43c8-4766-bb64-5522d0349eaa admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.108319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:37:59.123668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250e0d4-0a20-4e49-8f28-37abb010e968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.124453 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250e0d4-0a20-4e49-8f28-37abb010e968 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.124685 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250e0d4-0a20-4e49-8f28-37abb010e968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.125504 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250e0d4-0a20-4e49-8f28-37abb010e968 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.126258 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250e0d4-0a20-4e49-8f28-37abb010e968 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.126756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2fa50-deac-4e37-add9-bb6cd06d7c08 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.128208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250e0d4-0a20-4e49-8f28-37abb010e968 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.131316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aceb45-7d76-4ff7-9a33-a08f7c0d06d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.131993 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aceb45-7d76-4ff7-9a33-a08f7c0d06d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.132275 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aceb45-7d76-4ff7-9a33-a08f7c0d06d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.132544 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aceb45-7d76-4ff7-9a33-a08f7c0d06d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.133255 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aceb45-7d76-4ff7-9a33-a08f7c0d06d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.134819 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aceb45-7d76-4ff7-9a33-a08f7c0d06d0 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.135761 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:37:59.147633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 787/1571] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:37:59.148604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aceb45-7d76-4ff7-9a33-a08f7c0d06d0 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.148839 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 785/1572] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:37:59 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:37:59.149628 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250e0d4-0a20-4e49-8f28-37abb010e968 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.158386 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aceb45-7d76-4ff7-9a33-a08f7c0d06d0 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.160848 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250e0d4-0a20-4e49-8f28-37abb010e968 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.162899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3725a53-f6a5-4be4-b7a1-e1a4688b45f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.163537 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3725a53-f6a5-4be4-b7a1-e1a4688b45f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.163924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3725a53-f6a5-4be4-b7a1-e1a4688b45f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.164154 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3725a53-f6a5-4be4-b7a1-e1a4688b45f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.164831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3725a53-f6a5-4be4-b7a1-e1a4688b45f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.165542 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c7b98-109a-4111-9163-655c4840a667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.166175 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c7b98-109a-4111-9163-655c4840a667 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.166453 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c7b98-109a-4111-9163-655c4840a667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.166702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3725a53-f6a5-4be4-b7a1-e1a4688b45f5 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.166810 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c7b98-109a-4111-9163-655c4840a667 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.167435 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c7b98-109a-4111-9163-655c4840a667 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:37:59.167833 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c7b98-109a-4111-9163-655c4840a667 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.169327 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c7b98-109a-4111-9163-655c4840a667 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.182436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 786/1573] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:37:59.184629 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c7b98-109a-4111-9163-655c4840a667 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.194539 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 788/1574] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:37:59 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:37:59.195487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c7b98-109a-4111-9163-655c4840a667 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.195649 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3725a53-f6a5-4be4-b7a1-e1a4688b45f5 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.253194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ab6a6-7d9b-4cac-ae60-f746492c446a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.253862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ab6a6-7d9b-4cac-ae60-f746492c446a admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.254150 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ab6a6-7d9b-4cac-ae60-f746492c446a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.254428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ab6a6-7d9b-4cac-ae60-f746492c446a admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.255120 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ab6a6-7d9b-4cac-ae60-f746492c446a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.256433 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3725a53-f6a5-4be4-b7a1-e1a4688b45f5 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.257084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ab6a6-7d9b-4cac-ae60-f746492c446a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.260944 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a028d-ed44-47a8-b10c-4ba43b1cfa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.261598 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a028d-ed44-47a8-b10c-4ba43b1cfa98 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.261885 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a028d-ed44-47a8-b10c-4ba43b1cfa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.262163 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a028d-ed44-47a8-b10c-4ba43b1cfa98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.262876 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a028d-ed44-47a8-b10c-4ba43b1cfa98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.264495 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a028d-ed44-47a8-b10c-4ba43b1cfa98 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.276160 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 789/1575] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:37:59.277306 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a028d-ed44-47a8-b10c-4ba43b1cfa98 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.282143 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 787/1576] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:37:59 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:37:59.283107 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ab6a6-7d9b-4cac-ae60-f746492c446a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.289676 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a028d-ed44-47a8-b10c-4ba43b1cfa98 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.292877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ab6a6-7d9b-4cac-ae60-f746492c446a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.294280 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a3e94-d411-4b56-a198-c565f37a20b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.294925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a3e94-d411-4b56-a198-c565f37a20b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.295211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a3e94-d411-4b56-a198-c565f37a20b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.295494 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a3e94-d411-4b56-a198-c565f37a20b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.296209 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a3e94-d411-4b56-a198-c565f37a20b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.297617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd3e5c5-d27d-4cd9-9118-2b7b36ce71c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.298336 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3e5c5-d27d-4cd9-9118-2b7b36ce71c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.298461 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a3e94-d411-4b56-a198-c565f37a20b5 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.298618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3e5c5-d27d-4cd9-9118-2b7b36ce71c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.298887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3e5c5-d27d-4cd9-9118-2b7b36ce71c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.299548 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd3e5c5-d27d-4cd9-9118-2b7b36ce71c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.301441 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd3e5c5-d27d-4cd9-9118-2b7b36ce71c6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.322493 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 788/1577] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:37:59 2022] POST /identity/v3/users => generated 301 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:37:59.323374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd3e5c5-d27d-4cd9-9118-2b7b36ce71c6 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.326113 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 790/1578] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:37:59 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:37:59.326845 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a3e94-d411-4b56-a198-c565f37a20b5 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.334064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd3e5c5-d27d-4cd9-9118-2b7b36ce71c6 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.389212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a3e94-d411-4b56-a198-c565f37a20b5 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.394374 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2b359-d7d5-4e7b-ad9d-2fa4ef857036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.395041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2b359-d7d5-4e7b-ad9d-2fa4ef857036 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.395163 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2ab45-e2bf-43e4-a54d-3dc8a1897ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.395325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2b359-d7d5-4e7b-ad9d-2fa4ef857036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.395600 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2b359-d7d5-4e7b-ad9d-2fa4ef857036 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.395971 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2ab45-e2bf-43e4-a54d-3dc8a1897ba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.396507 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2ab45-e2bf-43e4-a54d-3dc8a1897ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.396615 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2b359-d7d5-4e7b-ad9d-2fa4ef857036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.396779 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2ab45-e2bf-43e4-a54d-3dc8a1897ba4 admin admin] PATH_INFO: `/v3/projects/fe30c756a2f84166ad23af1edf83157e/users/f6b7d82c999646b6b6b96d565f0e3f9a/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.398179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2b359-d7d5-4e7b-ad9d-2fa4ef857036 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.408678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 791/1579] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:37:59.409496 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2b359-d7d5-4e7b-ad9d-2fa4ef857036 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.412118 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2ab45-e2bf-43e4-a54d-3dc8a1897ba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe30c756a2f84166ad23af1edf83157e, user_id=f6b7d82c999646b6b6b96d565f0e3f9a, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.414921 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2ab45-e2bf-43e4-a54d-3dc8a1897ba4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.418991 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2b359-d7d5-4e7b-ad9d-2fa4ef857036 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.425399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f161f-0bbf-4730-881f-61e170b7a188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.426500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f161f-0bbf-4730-881f-61e170b7a188 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.426888 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f161f-0bbf-4730-881f-61e170b7a188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.427348 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f161f-0bbf-4730-881f-61e170b7a188 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.428555 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f161f-0bbf-4730-881f-61e170b7a188 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.428656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 789/1580] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:37:59 2022] PUT /identity/v3/projects/fe30c756a2f84166ad23af1edf83157e/users/f6b7d82c999646b6b6b96d565f0e3f9a/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:37:59.432571 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2ab45-e2bf-43e4-a54d-3dc8a1897ba4 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.432678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f161f-0bbf-4730-881f-61e170b7a188 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.459558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 792/1581] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:37:59 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:37:59.460489 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f161f-0bbf-4730-881f-61e170b7a188 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.465989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:37:59.471728 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:INSERT=9,keystone:SELECT=91 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:37:59.502640 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2ab45-e2bf-43e4-a54d-3dc8a1897ba4 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.525059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f161f-0bbf-4730-881f-61e170b7a188 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.530052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60a4fe-3cd8-4d5e-a1bb-e931409cba00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.530712 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60a4fe-3cd8-4d5e-a1bb-e931409cba00 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.530994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60a4fe-3cd8-4d5e-a1bb-e931409cba00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.532059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60a4fe-3cd8-4d5e-a1bb-e931409cba00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.532821 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e60a4fe-3cd8-4d5e-a1bb-e931409cba00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.534408 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e60a4fe-3cd8-4d5e-a1bb-e931409cba00 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.544757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 793/1582] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:37:59.545561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60a4fe-3cd8-4d5e-a1bb-e931409cba00 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.555014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60a4fe-3cd8-4d5e-a1bb-e931409cba00 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.559560 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5107895-30ed-42aa-853c-bd49e6a04379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.559695 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7294f55-0173-4667-a286-ec32711405a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.560318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7294f55-0173-4667-a286-ec32711405a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.560421 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5107895-30ed-42aa-853c-bd49e6a04379 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.560602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7294f55-0173-4667-a286-ec32711405a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.560722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5107895-30ed-42aa-853c-bd49e6a04379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.560875 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7294f55-0173-4667-a286-ec32711405a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.560979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5107895-30ed-42aa-853c-bd49e6a04379 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.561562 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7294f55-0173-4667-a286-ec32711405a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.561668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5107895-30ed-42aa-853c-bd49e6a04379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.563972 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5107895-30ed-42aa-853c-bd49e6a04379 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.564094 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7294f55-0173-4667-a286-ec32711405a0 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.573867 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 794/1583] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:37:59.577389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5107895-30ed-42aa-853c-bd49e6a04379 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.586175 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5107895-30ed-42aa-853c-bd49e6a04379 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.589434 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 790/1584] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:37:59 2022] POST /identity/v3/users => generated 296 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:37:59.589434 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7294f55-0173-4667-a286-ec32711405a0 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.590984 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed41dd-b6c7-43a9-b5f9-69819e52dd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.591649 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed41dd-b6c7-43a9-b5f9-69819e52dd96 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.591961 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed41dd-b6c7-43a9-b5f9-69819e52dd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.592233 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed41dd-b6c7-43a9-b5f9-69819e52dd96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.593427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed41dd-b6c7-43a9-b5f9-69819e52dd96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.595068 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed41dd-b6c7-43a9-b5f9-69819e52dd96 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.597782 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7294f55-0173-4667-a286-ec32711405a0 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.602551 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa26ee-e784-4c44-8e12-db44efa8ae09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.603308 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa26ee-e784-4c44-8e12-db44efa8ae09 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.603698 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa26ee-e784-4c44-8e12-db44efa8ae09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.604100 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa26ee-e784-4c44-8e12-db44efa8ae09 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.604968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa26ee-e784-4c44-8e12-db44efa8ae09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.606747 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 795/1585] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:37:59.610628 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa26ee-e784-4c44-8e12-db44efa8ae09 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.610828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed41dd-b6c7-43a9-b5f9-69819e52dd96 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.617779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed41dd-b6c7-43a9-b5f9-69819e52dd96 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.622891 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c5589-24cd-4dd3-a565-402123a8cdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.623525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c5589-24cd-4dd3-a565-402123a8cdac admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.623824 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c5589-24cd-4dd3-a565-402123a8cdac admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.624097 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c5589-24cd-4dd3-a565-402123a8cdac admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.624840 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c5589-24cd-4dd3-a565-402123a8cdac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.626398 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c5589-24cd-4dd3-a565-402123a8cdac admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.630660 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 791/1586] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:37:59 2022] POST /identity/v3/users => generated 287 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:37:59.630660 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e595b0-3faf-4750-9882-b6230400c49e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.630660 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e595b0-3faf-4750-9882-b6230400c49e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.630660 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e595b0-3faf-4750-9882-b6230400c49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.642726 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 796/1587] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:37:59.642726 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c5589-24cd-4dd3-a565-402123a8cdac admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.647733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-26e595b0-3faf-4750-9882-b6230400c49e None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:37:59.648967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-26e595b0-3faf-4750-9882-b6230400c49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:37:59.650403 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c5589-24cd-4dd3-a565-402123a8cdac admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.654887 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1baee88-f471-491e-9662-d0f692157efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.655531 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1baee88-f471-491e-9662-d0f692157efb admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.655841 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1baee88-f471-491e-9662-d0f692157efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.656803 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1baee88-f471-491e-9662-d0f692157efb admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.657103 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 792/1588] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:37:59.657323 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1baee88-f471-491e-9662-d0f692157efb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.657843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e595b0-3faf-4750-9882-b6230400c49e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.659295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1baee88-f471-491e-9662-d0f692157efb admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.667257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e595b0-3faf-4750-9882-b6230400c49e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.673103 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08e95f-d81a-4a90-83b2-6406f4d5d1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.674331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08e95f-d81a-4a90-83b2-6406f4d5d1cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.674610 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08e95f-d81a-4a90-83b2-6406f4d5d1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.674886 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08e95f-d81a-4a90-83b2-6406f4d5d1cc admin admin] PATH_INFO: `/v3/projects/0c5c564ff29441f786aa24ba5eb74f3e/users/b0149939e9dd4c2d94950a83419e64a9/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.680196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 797/1589] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:37:59 2022] POST /identity/v3/users => generated 293 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:37:59.681089 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1baee88-f471-491e-9662-d0f692157efb admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.691863 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08e95f-d81a-4a90-83b2-6406f4d5d1cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c5c564ff29441f786aa24ba5eb74f3e, user_id=b0149939e9dd4c2d94950a83419e64a9, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.692024 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1baee88-f471-491e-9662-d0f692157efb admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.694524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08e95f-d81a-4a90-83b2-6406f4d5d1cc admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.696982 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f0d0b3-6574-4b9e-a41a-27343f943dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.697643 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0d0b3-6574-4b9e-a41a-27343f943dad admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.697927 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0d0b3-6574-4b9e-a41a-27343f943dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.698226 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0d0b3-6574-4b9e-a41a-27343f943dad admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.699038 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f0d0b3-6574-4b9e-a41a-27343f943dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.700834 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f0d0b3-6574-4b9e-a41a-27343f943dad admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.707303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 793/1590] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:37:59 2022] PUT /identity/v3/projects/0c5c564ff29441f786aa24ba5eb74f3e/users/b0149939e9dd4c2d94950a83419e64a9/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:37:59.708137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08e95f-d81a-4a90-83b2-6406f4d5d1cc admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.722246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 798/1591] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:37:59.722980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f0d0b3-6574-4b9e-a41a-27343f943dad admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.732535 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f0d0b3-6574-4b9e-a41a-27343f943dad admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.769185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08e95f-d81a-4a90-83b2-6406f4d5d1cc admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.774228 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364893b7-4df5-4a68-aee9-6a159ff4e743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.774899 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364893b7-4df5-4a68-aee9-6a159ff4e743 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.775179 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364893b7-4df5-4a68-aee9-6a159ff4e743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.775453 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364893b7-4df5-4a68-aee9-6a159ff4e743 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.776213 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364893b7-4df5-4a68-aee9-6a159ff4e743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.778219 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364893b7-4df5-4a68-aee9-6a159ff4e743 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.788705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 794/1592] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:37:59.789493 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364893b7-4df5-4a68-aee9-6a159ff4e743 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.793131 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83f1d1-b178-4399-bc5d-45542e068598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.793803 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83f1d1-b178-4399-bc5d-45542e068598 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.794253 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83f1d1-b178-4399-bc5d-45542e068598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.794537 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83f1d1-b178-4399-bc5d-45542e068598 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.795259 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83f1d1-b178-4399-bc5d-45542e068598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.797652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83f1d1-b178-4399-bc5d-45542e068598 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.799346 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364893b7-4df5-4a68-aee9-6a159ff4e743 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.803938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecc84a-3054-41c3-be44-018517f95a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.804572 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecc84a-3054-41c3-be44-018517f95a88 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.804854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecc84a-3054-41c3-be44-018517f95a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.805129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecc84a-3054-41c3-be44-018517f95a88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.805863 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ecc84a-3054-41c3-be44-018517f95a88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.807545 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ecc84a-3054-41c3-be44-018517f95a88 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.811128 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 799/1593] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:37:59.811128 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83f1d1-b178-4399-bc5d-45542e068598 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.818083 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83f1d1-b178-4399-bc5d-45542e068598 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.823071 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 795/1594] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:37:59.823071 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecc84a-3054-41c3-be44-018517f95a88 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.825361 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65468738-96f1-4d7d-b5f2-644f2c91eb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.826227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65468738-96f1-4d7d-b5f2-644f2c91eb6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.826509 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65468738-96f1-4d7d-b5f2-644f2c91eb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.826924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65468738-96f1-4d7d-b5f2-644f2c91eb6a admin admin] PATH_INFO: `/v3/projects/8486d765afed40dc8795c7ba485d9535/users/3bbc3cd193444a65861f6e6705fd32cf/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.832088 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:37:59.841982 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65468738-96f1-4d7d-b5f2-644f2c91eb6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8486d765afed40dc8795c7ba485d9535, user_id=3bbc3cd193444a65861f6e6705fd32cf, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.843016 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: WARNING py.warnings [None req-65468738-96f1-4d7d-b5f2-644f2c91eb6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 10:37:59.843016 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: warnings.warn(msg) Jun 06 10:37:59.843016 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: Jun 06 10:37:59.845606 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65468738-96f1-4d7d-b5f2-644f2c91eb6a admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.859108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 800/1595] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:37:59 2022] PUT /identity/v3/projects/8486d765afed40dc8795c7ba485d9535/users/3bbc3cd193444a65861f6e6705fd32cf/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:37:59.860594 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65468738-96f1-4d7d-b5f2-644f2c91eb6a admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.874906 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:37:59.882369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecc84a-3054-41c3-be44-018517f95a88 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.926209 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65468738-96f1-4d7d-b5f2-644f2c91eb6a admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.930682 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b9de1-db42-45df-9f14-66b5246f73cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.931339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b9de1-db42-45df-9f14-66b5246f73cd admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.931622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b9de1-db42-45df-9f14-66b5246f73cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.931932 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b9de1-db42-45df-9f14-66b5246f73cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.932204 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e1e4a-f3dc-442b-8f7a-3c3b2da0ce11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.932644 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b9de1-db42-45df-9f14-66b5246f73cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.932847 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e1e4a-f3dc-442b-8f7a-3c3b2da0ce11 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.933131 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e1e4a-f3dc-442b-8f7a-3c3b2da0ce11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.933408 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e1e4a-f3dc-442b-8f7a-3c3b2da0ce11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.934256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b9de1-db42-45df-9f14-66b5246f73cd admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.942877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9e1e4a-f3dc-442b-8f7a-3c3b2da0ce11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.944324 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 801/1596] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:37:59.944440 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9e1e4a-f3dc-442b-8f7a-3c3b2da0ce11 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:37:59.945380 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd53931-c387-4987-91e7-bbe23425781f None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.945672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd53931-c387-4987-91e7-bbe23425781f None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.945947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd53931-c387-4987-91e7-bbe23425781f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.954625 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 796/1597] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:37:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:37:59.955471 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e1e4a-f3dc-442b-8f7a-3c3b2da0ce11 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:37:59.964572 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e1e4a-f3dc-442b-8f7a-3c3b2da0ce11 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:37:59.970670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e46469-eeae-4424-b034-ff7587cec894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:37:59.971558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e46469-eeae-4424-b034-ff7587cec894 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:37:59.971868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e46469-eeae-4424-b034-ff7587cec894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:37:59.972142 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e46469-eeae-4424-b034-ff7587cec894 admin admin] PATH_INFO: `/v3/projects/4905170c438f456182ba201316a64029/users/ea1482ba53554f71865cea46ffbac23c/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:37:59.974641 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-2bd53931-c387-4987-91e7-bbe23425781f None None] MFA Rules not processed for user `b0149939e9dd4c2d94950a83419e64a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:37:59.975897 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-2bd53931-c387-4987-91e7-bbe23425781f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:37:59.988905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e46469-eeae-4424-b034-ff7587cec894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4905170c438f456182ba201316a64029, user_id=ea1482ba53554f71865cea46ffbac23c, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:37:59.991553 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e46469-eeae-4424-b034-ff7587cec894 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:00.010559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 797/1598] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:37:59 2022] PUT /identity/v3/projects/4905170c438f456182ba201316a64029/users/ea1482ba53554f71865cea46ffbac23c/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:00.010559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e46469-eeae-4424-b034-ff7587cec894 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:00.058121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 802/1599] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:37:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:00.059170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd53931-c387-4987-91e7-bbe23425781f None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:00.073084 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd53931-c387-4987-91e7-bbe23425781f None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:00.093288 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e46469-eeae-4424-b034-ff7587cec894 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:00.100654 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af799bb-388e-4981-98b2-aedd4552ac0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:00.101496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af799bb-388e-4981-98b2-aedd4552ac0d admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:00.101782 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af799bb-388e-4981-98b2-aedd4552ac0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:00.102440 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af799bb-388e-4981-98b2-aedd4552ac0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:00.103244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af799bb-388e-4981-98b2-aedd4552ac0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:00.104899 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af799bb-388e-4981-98b2-aedd4552ac0d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:00.117770 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 798/1600] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:00.118720 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af799bb-388e-4981-98b2-aedd4552ac0d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:00.131014 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af799bb-388e-4981-98b2-aedd4552ac0d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:00.136599 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c70252-a2d5-42b2-9687-8d270bb729b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:00.137276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c70252-a2d5-42b2-9687-8d270bb729b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:00.137558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c70252-a2d5-42b2-9687-8d270bb729b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:00.137835 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c70252-a2d5-42b2-9687-8d270bb729b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:00.138586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c70252-a2d5-42b2-9687-8d270bb729b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:00.140224 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c70252-a2d5-42b2-9687-8d270bb729b0 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:00.151770 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 799/1601] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:00.153051 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84cdc7-18dc-43c0-810e-085cab0f4c6b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:00.153335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84cdc7-18dc-43c0-810e-085cab0f4c6b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:00.153719 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84cdc7-18dc-43c0-810e-085cab0f4c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:00.159511 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744467d-b859-4b2f-9377-eb293e58df7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:00.160543 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744467d-b859-4b2f-9377-eb293e58df7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:00.160840 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744467d-b859-4b2f-9377-eb293e58df7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:00.161116 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744467d-b859-4b2f-9377-eb293e58df7b admin admin] PATH_INFO: `/v3/projects/5c7cf5d7d30245dfbf08c63be3a28dc7/users/a4a21940483a4c009a5e19301cf6cafb/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:00.172236 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-5c84cdc7-18dc-43c0-810e-085cab0f4c6b None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:00.174246 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5c84cdc7-18dc-43c0-810e-085cab0f4c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:00.178089 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744467d-b859-4b2f-9377-eb293e58df7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c7cf5d7d30245dfbf08c63be3a28dc7, user_id=a4a21940483a4c009a5e19301cf6cafb, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:00.180814 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744467d-b859-4b2f-9377-eb293e58df7b admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:00.184586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 800/1602] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:00.185818 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84cdc7-18dc-43c0-810e-085cab0f4c6b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:00.197742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84cdc7-18dc-43c0-810e-085cab0f4c6b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:00.199970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 803/1603] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:00 2022] PUT /identity/v3/projects/5c7cf5d7d30245dfbf08c63be3a28dc7/users/a4a21940483a4c009a5e19301cf6cafb/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:00.205557 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be2e99-9409-4d91-9d13-39a50460e1f7 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:00.205858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be2e99-9409-4d91-9d13-39a50460e1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:00.206202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be2e99-9409-4d91-9d13-39a50460e1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:00.216147 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:00.226390 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-98be2e99-9409-4d91-9d13-39a50460e1f7 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:00.227713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-98be2e99-9409-4d91-9d13-39a50460e1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:00.257473 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:00.274923 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636ac76-e1ae-4d2c-925d-06dfeffbaff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:00.275338 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636ac76-e1ae-4d2c-925d-06dfeffbaff1 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:00.275618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636ac76-e1ae-4d2c-925d-06dfeffbaff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:00.275923 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636ac76-e1ae-4d2c-925d-06dfeffbaff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:00.276683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636ac76-e1ae-4d2c-925d-06dfeffbaff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:00.279307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636ac76-e1ae-4d2c-925d-06dfeffbaff1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:00.290927 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 801/1604] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:00.291583 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636ac76-e1ae-4d2c-925d-06dfeffbaff1 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:00.303048 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636ac76-e1ae-4d2c-925d-06dfeffbaff1 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:00.310315 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe6734-3e5a-4cd8-8735-6a51cdca0ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:00.310981 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe6734-3e5a-4cd8-8735-6a51cdca0ee5 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:00.311276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe6734-3e5a-4cd8-8735-6a51cdca0ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:00.311536 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe6734-3e5a-4cd8-8735-6a51cdca0ee5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:00.312826 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe6734-3e5a-4cd8-8735-6a51cdca0ee5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:00.314701 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe6734-3e5a-4cd8-8735-6a51cdca0ee5 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:00.324178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 804/1605] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:00.325908 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be2e99-9409-4d91-9d13-39a50460e1f7 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:00.342268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be2e99-9409-4d91-9d13-39a50460e1f7 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:00.348418 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf35a74-da57-43ef-a0d9-645d8e99704d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:00.349545 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 802/1606] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:00 2022] POST /identity/v3/projects => generated 403 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:00.349956 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf35a74-da57-43ef-a0d9-645d8e99704d admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:00.350184 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe6734-3e5a-4cd8-8735-6a51cdca0ee5 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:00.350299 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf35a74-da57-43ef-a0d9-645d8e99704d admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:00.350750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf35a74-da57-43ef-a0d9-645d8e99704d admin admin] PATH_INFO: `/v3/projects/e2b17b61b73b4e6d81dd8ffe9cf311ea/users/f269e146c844480c822347c3a38d103b/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:00.371898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf35a74-da57-43ef-a0d9-645d8e99704d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2b17b61b73b4e6d81dd8ffe9cf311ea, user_id=f269e146c844480c822347c3a38d103b, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:00.377415 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf35a74-da57-43ef-a0d9-645d8e99704d admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:00.403318 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 805/1607] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:00 2022] PUT /identity/v3/projects/e2b17b61b73b4e6d81dd8ffe9cf311ea/users/f269e146c844480c822347c3a38d103b/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:00.404402 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf35a74-da57-43ef-a0d9-645d8e99704d admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:00.412748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-edf35a74-da57-43ef-a0d9-645d8e99704d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:00.438088 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe6734-3e5a-4cd8-8735-6a51cdca0ee5 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:00.495879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf35a74-da57-43ef-a0d9-645d8e99704d admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:00.501680 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80156c31-e33a-4ce2-a8ad-be8f3fb3df72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:00.502766 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80156c31-e33a-4ce2-a8ad-be8f3fb3df72 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:00.503041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80156c31-e33a-4ce2-a8ad-be8f3fb3df72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:00.503326 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80156c31-e33a-4ce2-a8ad-be8f3fb3df72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:00.506855 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8690031-11eb-454f-ac6e-5e98402798f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:00.507852 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8690031-11eb-454f-ac6e-5e98402798f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:00.508324 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8690031-11eb-454f-ac6e-5e98402798f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:00.508656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8690031-11eb-454f-ac6e-5e98402798f6 admin admin] PATH_INFO: `/v3/projects/f6c01a13906d4ed68dea196921d6ca25/users/347ca57a31424bc0b765ea1266097de5/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:00.515669 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80156c31-e33a-4ce2-a8ad-be8f3fb3df72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:00.518063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80156c31-e33a-4ce2-a8ad-be8f3fb3df72 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:00.524930 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8690031-11eb-454f-ac6e-5e98402798f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6c01a13906d4ed68dea196921d6ca25, user_id=347ca57a31424bc0b765ea1266097de5, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:00.528564 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8690031-11eb-454f-ac6e-5e98402798f6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:00.530497 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 806/1608] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:00.532413 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e48da-d9d6-498b-9d80-d40eb33bcf20 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:00.533076 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e48da-d9d6-498b-9d80-d40eb33bcf20 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:00.533565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e48da-d9d6-498b-9d80-d40eb33bcf20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:00.555181 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 803/1609] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:00 2022] PUT /identity/v3/projects/f6c01a13906d4ed68dea196921d6ca25/users/347ca57a31424bc0b765ea1266097de5/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:00.556452 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8690031-11eb-454f-ac6e-5e98402798f6 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:00.558513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-7a8e48da-d9d6-498b-9d80-d40eb33bcf20 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:00.559971 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-7a8e48da-d9d6-498b-9d80-d40eb33bcf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:00.598419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:00.651437 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8690031-11eb-454f-ac6e-5e98402798f6 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:00.654000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 807/1610] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:00.654779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e48da-d9d6-498b-9d80-d40eb33bcf20 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:00.664066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c943a404-a564-47ae-aac5-5e03137c66d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:00.676133 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c943a404-a564-47ae-aac5-5e03137c66d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:00.678585 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c943a404-a564-47ae-aac5-5e03137c66d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:00.678993 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c943a404-a564-47ae-aac5-5e03137c66d9 admin admin] PATH_INFO: `/v3/projects/3428f9ec1f0444e189b0d2c465f59419/users/749aa2ca183140e2a86d6be9aac6b4a4/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:00.704798 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c943a404-a564-47ae-aac5-5e03137c66d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3428f9ec1f0444e189b0d2c465f59419, user_id=749aa2ca183140e2a86d6be9aac6b4a4, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:00.708700 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c943a404-a564-47ae-aac5-5e03137c66d9 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:00.747924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 804/1611] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:00 2022] PUT /identity/v3/projects/3428f9ec1f0444e189b0d2c465f59419/users/749aa2ca183140e2a86d6be9aac6b4a4/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:00.749318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c943a404-a564-47ae-aac5-5e03137c66d9 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:00.773293 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:00.807989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e48da-d9d6-498b-9d80-d40eb33bcf20 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:00.883237 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c943a404-a564-47ae-aac5-5e03137c66d9 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:00.893301 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81868588-a24f-458d-bc73-fcb9019a4c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:00.893999 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81868588-a24f-458d-bc73-fcb9019a4c1f admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:00.894283 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81868588-a24f-458d-bc73-fcb9019a4c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:00.894565 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81868588-a24f-458d-bc73-fcb9019a4c1f admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:00.895285 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81868588-a24f-458d-bc73-fcb9019a4c1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:00.898142 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81868588-a24f-458d-bc73-fcb9019a4c1f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:00.925995 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262ff77-df82-4a5a-a5d8-0c062f4299a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:00.926662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262ff77-df82-4a5a-a5d8-0c062f4299a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:00.926949 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262ff77-df82-4a5a-a5d8-0c062f4299a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:00.927219 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262ff77-df82-4a5a-a5d8-0c062f4299a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:00.928319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 805/1612] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:00 2022] POST /identity/v3/users => generated 308 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:00.929846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd578f1b-4eb3-466c-b525-31331c809717 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:00.930203 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd578f1b-4eb3-466c-b525-31331c809717 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:00.930615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd578f1b-4eb3-466c-b525-31331c809717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:00.947058 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262ff77-df82-4a5a-a5d8-0c062f4299a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:00.948290 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262ff77-df82-4a5a-a5d8-0c062f4299a3 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:00.951643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-dd578f1b-4eb3-466c-b525-31331c809717 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:00.953475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-dd578f1b-4eb3-466c-b525-31331c809717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:00.966873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 806/1613] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:00.970807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b146e0-1812-4ef0-9510-50d9fef96c7e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:00.971111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b146e0-1812-4ef0-9510-50d9fef96c7e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:00.971752 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b146e0-1812-4ef0-9510-50d9fef96c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:00.974577 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 808/1614] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:00.978713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262ff77-df82-4a5a-a5d8-0c062f4299a3 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:01.007357 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262ff77-df82-4a5a-a5d8-0c062f4299a3 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:01.013022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72884885-54fd-4eef-8e9d-acf1db2e08be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:01.013922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72884885-54fd-4eef-8e9d-acf1db2e08be service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:01.014272 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72884885-54fd-4eef-8e9d-acf1db2e08be service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:01.014623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72884885-54fd-4eef-8e9d-acf1db2e08be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:01.014977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-03b146e0-1812-4ef0-9510-50d9fef96c7e None None] MFA Rules not processed for user `347ca57a31424bc0b765ea1266097de5`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:01.016963 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-03b146e0-1812-4ef0-9510-50d9fef96c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:01.066143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72884885-54fd-4eef-8e9d-acf1db2e08be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:01.074837 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72884885-54fd-4eef-8e9d-acf1db2e08be service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:01.119045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 809/1615] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:01.124492 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018ad9f-02b0-484a-ad39-60c5688d2d5b None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:01.125482 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018ad9f-02b0-484a-ad39-60c5688d2d5b None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:01.126186 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 807/1616] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:01.127120 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b146e0-1812-4ef0-9510-50d9fef96c7e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:01.132253 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018ad9f-02b0-484a-ad39-60c5688d2d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:01.143647 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b146e0-1812-4ef0-9510-50d9fef96c7e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:01.150682 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7a79e-b3f1-4b92-b0b2-4f7ae54e77f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:01.152210 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7a79e-b3f1-4b92-b0b2-4f7ae54e77f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:01.152507 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7a79e-b3f1-4b92-b0b2-4f7ae54e77f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:01.152847 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7a79e-b3f1-4b92-b0b2-4f7ae54e77f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:01.153637 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7a79e-b3f1-4b92-b0b2-4f7ae54e77f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:01.155538 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7a79e-b3f1-4b92-b0b2-4f7ae54e77f4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:01.171470 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 808/1617] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:01.172880 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7a79e-b3f1-4b92-b0b2-4f7ae54e77f4 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:01.184619 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7a79e-b3f1-4b92-b0b2-4f7ae54e77f4 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:01.199099 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949db20b-4e01-46aa-b335-c53312359031 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:01.200333 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949db20b-4e01-46aa-b335-c53312359031 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:01.200858 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949db20b-4e01-46aa-b335-c53312359031 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:01.201375 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949db20b-4e01-46aa-b335-c53312359031 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:01.212950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:01.214990 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-3018ad9f-02b0-484a-ad39-60c5688d2d5b None None] MFA Rules not processed for user `749aa2ca183140e2a86d6be9aac6b4a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:01.216289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-3018ad9f-02b0-484a-ad39-60c5688d2d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:01.216526 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949db20b-4e01-46aa-b335-c53312359031 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:01.218275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949db20b-4e01-46aa-b335-c53312359031 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:01.231755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 809/1618] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:01.232907 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949db20b-4e01-46aa-b335-c53312359031 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:01.245615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949db20b-4e01-46aa-b335-c53312359031 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:01.253512 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c6e45-923f-4767-9b54-32dbea957d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:01.254391 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c6e45-923f-4767-9b54-32dbea957d76 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:01.255438 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c6e45-923f-4767-9b54-32dbea957d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:01.256017 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c6e45-923f-4767-9b54-32dbea957d76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:01.256835 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c6e45-923f-4767-9b54-32dbea957d76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:01.259118 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c6e45-923f-4767-9b54-32dbea957d76 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:01.307472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 810/1619] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:01.309327 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 810/1620] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:01 2022] POST /identity/v3/projects => generated 371 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:01.314775 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018ad9f-02b0-484a-ad39-60c5688d2d5b None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:01.315344 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c6e45-923f-4767-9b54-32dbea957d76 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:01.331031 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018ad9f-02b0-484a-ad39-60c5688d2d5b None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:01.442552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c6e45-923f-4767-9b54-32dbea957d76 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:01.457558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131315a-5ac5-4ebe-aac1-a53d509ce66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:01.457765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4fa43b-7429-4330-a574-ecb736e83e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:01.458305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131315a-5ac5-4ebe-aac1-a53d509ce66e admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:01.458616 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131315a-5ac5-4ebe-aac1-a53d509ce66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:01.458924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131315a-5ac5-4ebe-aac1-a53d509ce66e admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:01.459209 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fa43b-7429-4330-a574-ecb736e83e43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:01.459498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fa43b-7429-4330-a574-ecb736e83e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:01.459659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c131315a-5ac5-4ebe-aac1-a53d509ce66e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:01.459943 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fa43b-7429-4330-a574-ecb736e83e43 admin admin] PATH_INFO: `/v3/projects/cdc3fd148cde4c80a2304f9d9436b420/users/a9b7eabc06ce4c77a48b50a8c60307b1/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:01.476206 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c131315a-5ac5-4ebe-aac1-a53d509ce66e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:01.478427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4fa43b-7429-4330-a574-ecb736e83e43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdc3fd148cde4c80a2304f9d9436b420, user_id=a9b7eabc06ce4c77a48b50a8c60307b1, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:01.481686 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4fa43b-7429-4330-a574-ecb736e83e43 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:01.494340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:01.514196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 811/1621] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:01 2022] PUT /identity/v3/projects/cdc3fd148cde4c80a2304f9d9436b420/users/a9b7eabc06ce4c77a48b50a8c60307b1/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:01.522130 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 811/1622] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:01 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:01.523045 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131315a-5ac5-4ebe-aac1-a53d509ce66e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:01.523901 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f2728-7397-4d2a-b4ac-bb81e878cd00 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:01.524157 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f2728-7397-4d2a-b4ac-bb81e878cd00 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:01.524842 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f2728-7397-4d2a-b4ac-bb81e878cd00 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:01.526601 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 812/1623] 10.222.0.42 () {54 vars in 961 bytes} [Mon Jun 6 10:38:01 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:38:01.529049 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f2728-7397-4d2a-b4ac-bb81e878cd00 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:01.548827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131315a-5ac5-4ebe-aac1-a53d509ce66e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:01.661412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbccbb9-2032-435e-991e-e1444d300713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:01.662106 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbccbb9-2032-435e-991e-e1444d300713 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:01.662393 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbccbb9-2032-435e-991e-e1444d300713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:01.662677 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbccbb9-2032-435e-991e-e1444d300713 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:01.664417 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f2728-7397-4d2a-b4ac-bb81e878cd00 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:01.665506 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbccbb9-2032-435e-991e-e1444d300713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:01.667158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbccbb9-2032-435e-991e-e1444d300713 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:01.674773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfa955-42a9-482d-bab6-f338d12541e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:01.675617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfa955-42a9-482d-bab6-f338d12541e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:01.675930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfa955-42a9-482d-bab6-f338d12541e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:01.676297 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfa955-42a9-482d-bab6-f338d12541e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:01.676999 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdfa955-42a9-482d-bab6-f338d12541e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:01.678624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdfa955-42a9-482d-bab6-f338d12541e8 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:01.696845 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 812/1624] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:01.699967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae652d-10c2-4308-95ca-e3660bb68481 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:01.700286 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae652d-10c2-4308-95ca-e3660bb68481 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:01.706345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae652d-10c2-4308-95ca-e3660bb68481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:01.715063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 813/1625] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:01.717581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfa955-42a9-482d-bab6-f338d12541e8 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:01.739189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfa955-42a9-482d-bab6-f338d12541e8 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:01.748694 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-06ae652d-10c2-4308-95ca-e3660bb68481 None None] MFA Rules not processed for user `ed8a40e4f1034a8e99d0eb505e448db9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:01.749726 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-06ae652d-10c2-4308-95ca-e3660bb68481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:01.758541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed27d7-ea4a-483d-8d14-743be51bfc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:01.759470 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed27d7-ea4a-483d-8d14-743be51bfc30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:01.759753 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed27d7-ea4a-483d-8d14-743be51bfc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:01.760080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed27d7-ea4a-483d-8d14-743be51bfc30 admin admin] PATH_INFO: `/v3/projects/cdc3fd148cde4c80a2304f9d9436b420/users/a9b7eabc06ce4c77a48b50a8c60307b1/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:01.763883 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed27d7-ea4a-483d-8d14-743be51bfc30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdc3fd148cde4c80a2304f9d9436b420, user_id=a9b7eabc06ce4c77a48b50a8c60307b1, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:01.772159 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed27d7-ea4a-483d-8d14-743be51bfc30 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:01.808476 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 814/1626] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:01 2022] PUT /identity/v3/projects/cdc3fd148cde4c80a2304f9d9436b420/users/a9b7eabc06ce4c77a48b50a8c60307b1/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:01.819850 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed27d7-ea4a-483d-8d14-743be51bfc30 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:01.919914 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed27d7-ea4a-483d-8d14-743be51bfc30 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:01.929040 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:01.932991 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e12cc-b34a-4225-b22e-5e4f668c0407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:01.934923 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e12cc-b34a-4225-b22e-5e4f668c0407 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:01.935297 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e12cc-b34a-4225-b22e-5e4f668c0407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:01.935885 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e12cc-b34a-4225-b22e-5e4f668c0407 admin admin] PATH_INFO: `/v3/projects/dda2ef0ed9234862bcf3af1f94b1525d/users/458f2462a7744850ab2179028d039dd2/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:01.969408 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 813/1627] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Jun 6 10:38:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:01.970963 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06955a9-72d4-4a6d-b8f6-e72810e51dc3 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:01.971255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06955a9-72d4-4a6d-b8f6-e72810e51dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:01.971537 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06955a9-72d4-4a6d-b8f6-e72810e51dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:01.980209 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e12cc-b34a-4225-b22e-5e4f668c0407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dda2ef0ed9234862bcf3af1f94b1525d, user_id=458f2462a7744850ab2179028d039dd2, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:01.983644 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e12cc-b34a-4225-b22e-5e4f668c0407 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:02.013729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 815/1628] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:01 2022] PUT /identity/v3/projects/dda2ef0ed9234862bcf3af1f94b1525d/users/458f2462a7744850ab2179028d039dd2/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:02.014680 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e12cc-b34a-4225-b22e-5e4f668c0407 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:02.036573 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b06955a9-72d4-4a6d-b8f6-e72810e51dc3 None None] MFA Rules not processed for user `a9b7eabc06ce4c77a48b50a8c60307b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:02.044285 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b06955a9-72d4-4a6d-b8f6-e72810e51dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:02.186141 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:02.204827 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e12cc-b34a-4225-b22e-5e4f668c0407 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:02.224445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 814/1629] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:01 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:02.227487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06955a9-72d4-4a6d-b8f6-e72810e51dc3 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:02.238440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f994106f-e8c1-4bd1-962a-700da0637771 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:02.239771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994106f-e8c1-4bd1-962a-700da0637771 service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:02.240415 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994106f-e8c1-4bd1-962a-700da0637771 service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:02.241335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994106f-e8c1-4bd1-962a-700da0637771 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:02.357893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f994106f-e8c1-4bd1-962a-700da0637771 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:02.360830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f994106f-e8c1-4bd1-962a-700da0637771 service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:02.371330 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 816/1630] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:02.372201 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f994106f-e8c1-4bd1-962a-700da0637771 service nova] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:02.438565 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06955a9-72d4-4a6d-b8f6-e72810e51dc3 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:02.449510 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb36fb-26fe-42cc-9f4e-eddf219bbad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:02.451067 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb36fb-26fe-42cc-9f4e-eddf219bbad6 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:02.451420 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb36fb-26fe-42cc-9f4e-eddf219bbad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:02.452448 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb36fb-26fe-42cc-9f4e-eddf219bbad6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:02.453908 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb36fb-26fe-42cc-9f4e-eddf219bbad6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:02.455975 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb36fb-26fe-42cc-9f4e-eddf219bbad6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:02.468874 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 815/1631] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:02.489283 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb36fb-26fe-42cc-9f4e-eddf219bbad6 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:02.522619 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb36fb-26fe-42cc-9f4e-eddf219bbad6 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:02.525574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f994106f-e8c1-4bd1-962a-700da0637771 service nova] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:02.532848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1768d-b706-4c36-aeec-19afb73c5b29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:02.533059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1768d-b706-4c36-aeec-19afb73c5b29 service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:02.533404 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1768d-b706-4c36-aeec-19afb73c5b29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:02.533799 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1768d-b706-4c36-aeec-19afb73c5b29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:02.536497 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a216b-e25d-49c0-aa69-133c9b1ff26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:02.541707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a216b-e25d-49c0-aa69-133c9b1ff26b admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:02.542052 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a216b-e25d-49c0-aa69-133c9b1ff26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:02.542314 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a216b-e25d-49c0-aa69-133c9b1ff26b admin admin] PATH_INFO: `/v3/projects/dda2ef0ed9234862bcf3af1f94b1525d/users/458f2462a7744850ab2179028d039dd2/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:02.551448 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a216b-e25d-49c0-aa69-133c9b1ff26b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dda2ef0ed9234862bcf3af1f94b1525d, user_id=458f2462a7744850ab2179028d039dd2, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:02.552585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1768d-b706-4c36-aeec-19afb73c5b29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:02.554248 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a216b-e25d-49c0-aa69-133c9b1ff26b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:02.554392 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1768d-b706-4c36-aeec-19afb73c5b29 service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:02.581846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 816/1632] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:02 2022] PUT /identity/v3/projects/dda2ef0ed9234862bcf3af1f94b1525d/users/458f2462a7744850ab2179028d039dd2/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:02.592323 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff7d55-6482-48b2-aa3a-2c154d5f65ab None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:02.592604 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff7d55-6482-48b2-aa3a-2c154d5f65ab None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:02.592847 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 817/1633] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:02 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:02.593163 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff7d55-6482-48b2-aa3a-2c154d5f65ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:02.659584 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-f6ff7d55-6482-48b2-aa3a-2c154d5f65ab None None] MFA Rules not processed for user `458f2462a7744850ab2179028d039dd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:02.661708 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f6ff7d55-6482-48b2-aa3a-2c154d5f65ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:02.803966 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:02.862941 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 817/1634] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:02 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:02.877165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1768d-b706-4c36-aeec-19afb73c5b29 service cinder] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:02.898516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d28eef-46ec-4290-ab06-2520445d2444 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:02.898803 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d28eef-46ec-4290-ab06-2520445d2444 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:02.899723 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d28eef-46ec-4290-ab06-2520445d2444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:02.900600 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1768d-b706-4c36-aeec-19afb73c5b29 service cinder] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:03.002159 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-73d28eef-46ec-4290-ab06-2520445d2444 None None] MFA Rules not processed for user `3bbc3cd193444a65861f6e6705fd32cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:03.004293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-73d28eef-46ec-4290-ab06-2520445d2444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:03.029774 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a09080-b012-40b5-aa36-4252f554763b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:03.032403 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a09080-b012-40b5-aa36-4252f554763b service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:03.032703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a09080-b012-40b5-aa36-4252f554763b service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:03.033872 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a09080-b012-40b5-aa36-4252f554763b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:03.047997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a09080-b012-40b5-aa36-4252f554763b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:03.049534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a09080-b012-40b5-aa36-4252f554763b service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:03.086156 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 818/1635] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:02 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:03.089213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbecf40-1d6e-44ab-a032-40bef7386ddf None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:03.089977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbecf40-1d6e-44ab-a032-40bef7386ddf None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:03.090533 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbecf40-1d6e-44ab-a032-40bef7386ddf None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:03.105235 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 819/1636] 10.222.0.42 () {54 vars in 912 bytes} [Mon Jun 6 10:38:03 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:38:03.109096 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbecf40-1d6e-44ab-a032-40bef7386ddf None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:03.132864 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 818/1637] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:38:03.134412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d464d78-5e3e-4d10-acce-70a30902d29f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:03.134800 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d464d78-5e3e-4d10-acce-70a30902d29f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:03.135174 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d464d78-5e3e-4d10-acce-70a30902d29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:03.184987 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3d464d78-5e3e-4d10-acce-70a30902d29f None None] MFA Rules not processed for user `ed8a40e4f1034a8e99d0eb505e448db9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:03.186301 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3d464d78-5e3e-4d10-acce-70a30902d29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:03.228909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:03.264580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbecf40-1d6e-44ab-a032-40bef7386ddf None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:03.272264 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19eed4-101e-4b81-b147-e6eb41c5e1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:03.273705 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19eed4-101e-4b81-b147-e6eb41c5e1f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:03.274200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19eed4-101e-4b81-b147-e6eb41c5e1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:03.274698 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19eed4-101e-4b81-b147-e6eb41c5e1f8 admin admin] PATH_INFO: `/v3/users/b0149939e9dd4c2d94950a83419e64a9` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:03.276284 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19eed4-101e-4b81-b147-e6eb41c5e1f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0149939e9dd4c2d94950a83419e64a9)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:03.277933 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: WARNING py.warnings [None req-0d19eed4-101e-4b81-b147-e6eb41c5e1f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 10:38:03.277933 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: warnings.warn(msg) Jun 06 10:38:03.277933 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: Jun 06 10:38:03.282119 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19eed4-101e-4b81-b147-e6eb41c5e1f8 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:03.404150 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 819/1638] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jun 6 10:38:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:03.407018 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202015c9-911f-45c8-b00a-32f4daabb90c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:03.410237 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202015c9-911f-45c8-b00a-32f4daabb90c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:03.410759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202015c9-911f-45c8-b00a-32f4daabb90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:03.434537 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-0d19eed4-101e-4b81-b147-e6eb41c5e1f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0149939e9dd4c2d94950a83419e64a9'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:03.440791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-202015c9-911f-45c8-b00a-32f4daabb90c None None] MFA Rules not processed for user `3bbc3cd193444a65861f6e6705fd32cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:03.445896 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-202015c9-911f-45c8-b00a-32f4daabb90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:03.465519 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-0d19eed4-101e-4b81-b147-e6eb41c5e1f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0149939e9dd4c2d94950a83419e64a9'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:03.468096 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 820/1639] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:03.469260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202015c9-911f-45c8-b00a-32f4daabb90c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:03.487422 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202015c9-911f-45c8-b00a-32f4daabb90c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:03.492641 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-0d19eed4-101e-4b81-b147-e6eb41c5e1f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0149939e9dd4c2d94950a83419e64a9'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:03.493334 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-0d19eed4-101e-4b81-b147-e6eb41c5e1f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0149939e9dd4c2d94950a83419e64a9'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:03.502583 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d018c-204c-40dd-83d2-99fb0fa392a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:03.505952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d018c-204c-40dd-83d2-99fb0fa392a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:03.506296 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d018c-204c-40dd-83d2-99fb0fa392a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:03.507156 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d018c-204c-40dd-83d2-99fb0fa392a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:03.540681 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d018c-204c-40dd-83d2-99fb0fa392a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:03.544022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d018c-204c-40dd-83d2-99fb0fa392a3 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:03.553203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-0d19eed4-101e-4b81-b147-e6eb41c5e1f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0149939e9dd4c2d94950a83419e64a9'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:03.569220 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 821/1640] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:03.571226 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815c010-afe0-4f82-9398-68836de366d2 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:03.571612 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 820/1641] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:03 2022] DELETE /identity/v3/users/b0149939e9dd4c2d94950a83419e64a9 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:03.575726 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815c010-afe0-4f82-9398-68836de366d2 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:03.576077 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815c010-afe0-4f82-9398-68836de366d2 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:03.577061 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e9618-1bf4-41db-917a-94cfeb6df54f None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:03.577359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e9618-1bf4-41db-917a-94cfeb6df54f None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:03.577688 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e9618-1bf4-41db-917a-94cfeb6df54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:03.583700 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 822/1642] 10.222.0.42 () {54 vars in 961 bytes} [Mon Jun 6 10:38:03 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:38:03.585367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00acf2a5-f585-4c03-97a5-be2d4b675b29 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:03.585653 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00acf2a5-f585-4c03-97a5-be2d4b675b29 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:03.585926 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00acf2a5-f585-4c03-97a5-be2d4b675b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:03.618692 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-00acf2a5-f585-4c03-97a5-be2d4b675b29 None None] MFA Rules not processed for user `ed8a40e4f1034a8e99d0eb505e448db9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:03.619976 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-00acf2a5-f585-4c03-97a5-be2d4b675b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:03.628119 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-dd3e9618-1bf4-41db-917a-94cfeb6df54f None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:03.629387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-dd3e9618-1bf4-41db-917a-94cfeb6df54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:03.735346 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:03.796690 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 821/1643] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:03.807512 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 823/1644] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Jun 6 10:38:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:03.810883 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e9618-1bf4-41db-917a-94cfeb6df54f None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:03.812480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-dd3e9618-1bf4-41db-917a-94cfeb6df54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:03.816889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00acf2a5-f585-4c03-97a5-be2d4b675b29 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:03.845618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00acf2a5-f585-4c03-97a5-be2d4b675b29 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:03.866297 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45948c78-c616-472b-a84f-67390c8b65c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:03.867097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45948c78-c616-472b-a84f-67390c8b65c9 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:03.867485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45948c78-c616-472b-a84f-67390c8b65c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:03.867950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45948c78-c616-472b-a84f-67390c8b65c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:03.873169 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-45948c78-c616-472b-a84f-67390c8b65c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:03.946355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e9618-1bf4-41db-917a-94cfeb6df54f None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:03.963106 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046bcca5-b5b7-46b3-b64a-647034b305a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:03.964281 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bcca5-b5b7-46b3-b64a-647034b305a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:03.964577 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bcca5-b5b7-46b3-b64a-647034b305a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:03.965437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bcca5-b5b7-46b3-b64a-647034b305a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:03.991878 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046bcca5-b5b7-46b3-b64a-647034b305a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:03.993597 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046bcca5-b5b7-46b3-b64a-647034b305a3 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:04.010235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45948c78-c616-472b-a84f-67390c8b65c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:04.012203 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45948c78-c616-472b-a84f-67390c8b65c9 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:04.022074 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 822/1645] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:04.022990 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046bcca5-b5b7-46b3-b64a-647034b305a3 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:04.026852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-046bcca5-b5b7-46b3-b64a-647034b305a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:04.029950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 824/1646] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:04.045763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046bcca5-b5b7-46b3-b64a-647034b305a3 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:04.052635 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30581094-a759-4fa3-b222-539a4e1efe82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:04.054026 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30581094-a759-4fa3-b222-539a4e1efe82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:04.054524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30581094-a759-4fa3-b222-539a4e1efe82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:04.055042 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30581094-a759-4fa3-b222-539a4e1efe82 admin admin] PATH_INFO: `/v3/users/347ca57a31424bc0b765ea1266097de5` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:04.058674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30581094-a759-4fa3-b222-539a4e1efe82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=347ca57a31424bc0b765ea1266097de5)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:04.061579 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30581094-a759-4fa3-b222-539a4e1efe82 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:04.068609 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:04.155200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-30581094-a759-4fa3-b222-539a4e1efe82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '347ca57a31424bc0b765ea1266097de5'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:04.170144 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-30581094-a759-4fa3-b222-539a4e1efe82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '347ca57a31424bc0b765ea1266097de5'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:04.184254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-30581094-a759-4fa3-b222-539a4e1efe82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '347ca57a31424bc0b765ea1266097de5'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:04.184925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-30581094-a759-4fa3-b222-539a4e1efe82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '347ca57a31424bc0b765ea1266097de5'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:04.205910 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-30581094-a759-4fa3-b222-539a4e1efe82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '347ca57a31424bc0b765ea1266097de5'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:04.219855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 823/1647] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:04 2022] DELETE /identity/v3/users/347ca57a31424bc0b765ea1266097de5 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:04.221450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c9496-2efb-4781-b554-2b2a4cf664a6 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:04.221608 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c9496-2efb-4781-b554-2b2a4cf664a6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:04.221836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c9496-2efb-4781-b554-2b2a4cf664a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:04.243043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-954c9496-2efb-4781-b554-2b2a4cf664a6 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:04.245566 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-954c9496-2efb-4781-b554-2b2a4cf664a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:04.353681 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 825/1648] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:38:04.371185 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30581094-a759-4fa3-b222-539a4e1efe82 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:04.372901 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-30581094-a759-4fa3-b222-539a4e1efe82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:04.497407 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30581094-a759-4fa3-b222-539a4e1efe82 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:04.522305 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571f3e0-fdd8-4cb2-9856-9d6037411d33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:04.522972 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571f3e0-fdd8-4cb2-9856-9d6037411d33 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:04.523254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571f3e0-fdd8-4cb2-9856-9d6037411d33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:04.523527 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571f3e0-fdd8-4cb2-9856-9d6037411d33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:04.529908 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db64d2b-1bba-4df1-a4a3-c261588adc4c None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:04.530416 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db64d2b-1bba-4df1-a4a3-c261588adc4c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:04.530499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db64d2b-1bba-4df1-a4a3-c261588adc4c None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:04.531962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 826/1649] 10.222.0.42 () {54 vars in 912 bytes} [Mon Jun 6 10:38:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:38:04.543170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571f3e0-fdd8-4cb2-9856-9d6037411d33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:04.545549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0a57b-6d39-4d7d-b617-b39450bf4345 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:04.545848 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0a57b-6d39-4d7d-b617-b39450bf4345 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:04.546120 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0a57b-6d39-4d7d-b617-b39450bf4345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:04.558013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571f3e0-fdd8-4cb2-9856-9d6037411d33 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:04.590584 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-19e0a57b-6d39-4d7d-b617-b39450bf4345 None None] MFA Rules not processed for user `ed8a40e4f1034a8e99d0eb505e448db9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:04.591701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 824/1650] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:04.593828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-19e0a57b-6d39-4d7d-b617-b39450bf4345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:04.807526 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 827/1651] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jun 6 10:38:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:38:04.830177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571f3e0-fdd8-4cb2-9856-9d6037411d33 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:04.831589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-0571f3e0-fdd8-4cb2-9856-9d6037411d33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:05.002499 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571f3e0-fdd8-4cb2-9856-9d6037411d33 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:05.015996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1bd18-18e9-41ff-995a-4048290e522f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a67e007fc314b34aac2471a7c20f240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:05.017131 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1bd18-18e9-41ff-995a-4048290e522f service glance] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:05.017666 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1bd18-18e9-41ff-995a-4048290e522f service glance] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:05.018134 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1bd18-18e9-41ff-995a-4048290e522f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:05.038948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1bd18-18e9-41ff-995a-4048290e522f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:05.040987 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1bd18-18e9-41ff-995a-4048290e522f service glance] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:05.070058 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 825/1652] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:05.076839 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0a57b-6d39-4d7d-b617-b39450bf4345 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:05.078642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-19e0a57b-6d39-4d7d-b617-b39450bf4345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:05.100771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0a57b-6d39-4d7d-b617-b39450bf4345 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:05.111752 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e554a-e190-425d-9ec1-f0eae2cfc4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:05.115816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e554a-e190-425d-9ec1-f0eae2cfc4fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:05.116108 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e554a-e190-425d-9ec1-f0eae2cfc4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:05.116418 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e554a-e190-425d-9ec1-f0eae2cfc4fa admin admin] PATH_INFO: `/v3/projects/0c5c564ff29441f786aa24ba5eb74f3e` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:05.120465 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e554a-e190-425d-9ec1-f0eae2cfc4fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c5c564ff29441f786aa24ba5eb74f3e)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:05.128197 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: WARNING py.warnings [None req-8c3e554a-e190-425d-9ec1-f0eae2cfc4fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 10:38:05.128197 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: warnings.warn(msg) Jun 06 10:38:05.128197 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: Jun 06 10:38:05.129490 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e554a-e190-425d-9ec1-f0eae2cfc4fa admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:05.135466 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:05.274047 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8c3e554a-e190-425d-9ec1-f0eae2cfc4fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c5c564ff29441f786aa24ba5eb74f3e', 'request_id': 'req-8c3e554a-e190-425d-9ec1-f0eae2cfc4fa'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:05.294092 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8c3e554a-e190-425d-9ec1-f0eae2cfc4fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c5c564ff29441f786aa24ba5eb74f3e', 'request_id': 'req-8c3e554a-e190-425d-9ec1-f0eae2cfc4fa'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:05.307618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8c3e554a-e190-425d-9ec1-f0eae2cfc4fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c5c564ff29441f786aa24ba5eb74f3e', 'request_id': 'req-8c3e554a-e190-425d-9ec1-f0eae2cfc4fa'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:05.357301 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8c3e554a-e190-425d-9ec1-f0eae2cfc4fa admin admin] The token cache is being invalidate because project 0c5c564ff29441f786aa24ba5eb74f3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:05.357589 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8c3e554a-e190-425d-9ec1-f0eae2cfc4fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:05.360239 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8c3e554a-e190-425d-9ec1-f0eae2cfc4fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:05.362671 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 828/1653] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:05 2022] DELETE /identity/v3/projects/0c5c564ff29441f786aa24ba5eb74f3e => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:05.366454 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1bd18-18e9-41ff-995a-4048290e522f service glance] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:05.379654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1bd18-18e9-41ff-995a-4048290e522f service glance] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:05.463081 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:05.541542 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de505c-4bec-460b-b72c-2e756ca710bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:05.542403 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de505c-4bec-460b-b72c-2e756ca710bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:05.543449 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de505c-4bec-460b-b72c-2e756ca710bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:05.543449 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de505c-4bec-460b-b72c-2e756ca710bc admin admin] PATH_INFO: `/v3/users/a9b7eabc06ce4c77a48b50a8c60307b1` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:05.544431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de505c-4bec-460b-b72c-2e756ca710bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9b7eabc06ce4c77a48b50a8c60307b1)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:05.547467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de505c-4bec-460b-b72c-2e756ca710bc admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:05.648037 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-79de505c-4bec-460b-b72c-2e756ca710bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9b7eabc06ce4c77a48b50a8c60307b1'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:05.673139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-79de505c-4bec-460b-b72c-2e756ca710bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9b7eabc06ce4c77a48b50a8c60307b1'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:05.684619 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-79de505c-4bec-460b-b72c-2e756ca710bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9b7eabc06ce4c77a48b50a8c60307b1'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:05.718666 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-79de505c-4bec-460b-b72c-2e756ca710bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9b7eabc06ce4c77a48b50a8c60307b1'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:05.718666 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-79de505c-4bec-460b-b72c-2e756ca710bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9b7eabc06ce4c77a48b50a8c60307b1'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:05.729763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 826/1654] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:05 2022] DELETE /identity/v3/users/a9b7eabc06ce4c77a48b50a8c60307b1 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:05.865899 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316d0d6-a8c4-47fc-8e37-4b77653d8ea4 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:05.866484 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316d0d6-a8c4-47fc-8e37-4b77653d8ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:05.866484 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316d0d6-a8c4-47fc-8e37-4b77653d8ea4 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:05.869163 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 829/1655] 10.222.0.42 () {54 vars in 912 bytes} [Mon Jun 6 10:38:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:38:05.879421 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ddc92-ed99-4c96-b654-e78ac2dd1d77 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:05.880135 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ddc92-ed99-4c96-b654-e78ac2dd1d77 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:05.880135 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ddc92-ed99-4c96-b654-e78ac2dd1d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:05.963511 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316d0d6-a8c4-47fc-8e37-4b77653d8ea4 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:05.965609 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b316d0d6-a8c4-47fc-8e37-4b77653d8ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:06.021780 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-033ddc92-ed99-4c96-b654-e78ac2dd1d77 None None] MFA Rules not processed for user `881069f9b40a485dac2dda684dd6cd3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:06.026922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-033ddc92-ed99-4c96-b654-e78ac2dd1d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:06.215986 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316d0d6-a8c4-47fc-8e37-4b77653d8ea4 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:06.232168 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 827/1656] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jun 6 10:38:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:06.233628 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f460c6b-762b-49b9-b5b5-ee448e51bc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:06.235995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f460c6b-762b-49b9-b5b5-ee448e51bc82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:06.236279 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f460c6b-762b-49b9-b5b5-ee448e51bc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:06.240530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece83982-d74b-42a2-b8ec-9f7346dd7876 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:06.240823 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece83982-d74b-42a2-b8ec-9f7346dd7876 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:06.241104 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece83982-d74b-42a2-b8ec-9f7346dd7876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:06.242757 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f460c6b-762b-49b9-b5b5-ee448e51bc82 admin admin] PATH_INFO: `/v3/projects/f6c01a13906d4ed68dea196921d6ca25` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:06.245439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f460c6b-762b-49b9-b5b5-ee448e51bc82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6c01a13906d4ed68dea196921d6ca25)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:06.253089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f460c6b-762b-49b9-b5b5-ee448e51bc82 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:06.339140 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-ece83982-d74b-42a2-b8ec-9f7346dd7876 None None] MFA Rules not processed for user `a4a21940483a4c009a5e19301cf6cafb`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:06.340397 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ece83982-d74b-42a2-b8ec-9f7346dd7876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:06.373191 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3f460c6b-762b-49b9-b5b5-ee448e51bc82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6c01a13906d4ed68dea196921d6ca25', 'request_id': 'req-3f460c6b-762b-49b9-b5b5-ee448e51bc82'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:06.393069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3f460c6b-762b-49b9-b5b5-ee448e51bc82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6c01a13906d4ed68dea196921d6ca25', 'request_id': 'req-3f460c6b-762b-49b9-b5b5-ee448e51bc82'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:06.408476 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3f460c6b-762b-49b9-b5b5-ee448e51bc82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6c01a13906d4ed68dea196921d6ca25', 'request_id': 'req-3f460c6b-762b-49b9-b5b5-ee448e51bc82'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:06.418356 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:06.432563 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3f460c6b-762b-49b9-b5b5-ee448e51bc82 admin admin] The token cache is being invalidate because project f6c01a13906d4ed68dea196921d6ca25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:06.432868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3f460c6b-762b-49b9-b5b5-ee448e51bc82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:06.437639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3f460c6b-762b-49b9-b5b5-ee448e51bc82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:06.443557 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 830/1657] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:05 2022] DELETE /identity/v3/projects/f6c01a13906d4ed68dea196921d6ca25 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:38:06.444627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f460c6b-762b-49b9-b5b5-ee448e51bc82 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:06.448666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3f460c6b-762b-49b9-b5b5-ee448e51bc82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:06.529583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 828/1658] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:06.529583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece83982-d74b-42a2-b8ec-9f7346dd7876 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:06.529583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ece83982-d74b-42a2-b8ec-9f7346dd7876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:06.544871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:06.649356 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece83982-d74b-42a2-b8ec-9f7346dd7876 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:06.649578 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f460c6b-762b-49b9-b5b5-ee448e51bc82 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:06.655479 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a339c-ce97-4635-9d00-4071973e2727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:06.656872 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a339c-ce97-4635-9d00-4071973e2727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:06.657154 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a339c-ce97-4635-9d00-4071973e2727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:06.657426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a339c-ce97-4635-9d00-4071973e2727 admin admin] PATH_INFO: `/v3/users/458f2462a7744850ab2179028d039dd2` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:06.658996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a339c-ce97-4635-9d00-4071973e2727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=458f2462a7744850ab2179028d039dd2)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:06.660984 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a339c-ce97-4635-9d00-4071973e2727 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:06.663665 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33ab9d-c5b7-454e-9217-6e627065a41d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:06.664482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33ab9d-c5b7-454e-9217-6e627065a41d service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:06.664777 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33ab9d-c5b7-454e-9217-6e627065a41d service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:06.665064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33ab9d-c5b7-454e-9217-6e627065a41d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:06.668058 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ab33ab9d-c5b7-454e-9217-6e627065a41d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:06.698467 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33ab9d-c5b7-454e-9217-6e627065a41d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:06.700255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33ab9d-c5b7-454e-9217-6e627065a41d service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:06.734387 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 831/1659] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:06.735710 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33ab9d-c5b7-454e-9217-6e627065a41d service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:06.738642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ab33ab9d-c5b7-454e-9217-6e627065a41d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:06.770713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-a21a339c-ce97-4635-9d00-4071973e2727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '458f2462a7744850ab2179028d039dd2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:06.779680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33ab9d-c5b7-454e-9217-6e627065a41d service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:06.789000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-a21a339c-ce97-4635-9d00-4071973e2727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '458f2462a7744850ab2179028d039dd2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:06.793697 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9809003-9ad9-4f45-a9ab-c7324e0af732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:06.794431 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9809003-9ad9-4f45-a9ab-c7324e0af732 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:06.794717 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9809003-9ad9-4f45-a9ab-c7324e0af732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:06.795146 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9809003-9ad9-4f45-a9ab-c7324e0af732 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:06.797518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9809003-9ad9-4f45-a9ab-c7324e0af732 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:06.802914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9809003-9ad9-4f45-a9ab-c7324e0af732 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:06.807825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-a21a339c-ce97-4635-9d00-4071973e2727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '458f2462a7744850ab2179028d039dd2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:06.808959 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-a21a339c-ce97-4635-9d00-4071973e2727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '458f2462a7744850ab2179028d039dd2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:06.828324 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-a21a339c-ce97-4635-9d00-4071973e2727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '458f2462a7744850ab2179028d039dd2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:06.839833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 829/1660] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:06 2022] DELETE /identity/v3/users/458f2462a7744850ab2179028d039dd2 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:06.862406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 832/1661] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:06 2022] POST /identity/v3/projects => generated 403 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:06.866174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a339c-ce97-4635-9d00-4071973e2727 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:06.867900 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a21a339c-ce97-4635-9d00-4071973e2727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:06.959049 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:06.978291 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a339c-ce97-4635-9d00-4071973e2727 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:07.069023 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b23408-74b6-4f78-bd56-0d15a9a509ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:07.069867 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b23408-74b6-4f78-bd56-0d15a9a509ae admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:07.070221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b23408-74b6-4f78-bd56-0d15a9a509ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:07.070562 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b23408-74b6-4f78-bd56-0d15a9a509ae admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:07.071995 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b23408-74b6-4f78-bd56-0d15a9a509ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:07.074825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b23408-74b6-4f78-bd56-0d15a9a509ae admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:07.109883 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 830/1662] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:06 2022] POST /identity/v3/users => generated 308 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:07.113251 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9809003-9ad9-4f45-a9ab-c7324e0af732 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:07.132168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9809003-9ad9-4f45-a9ab-c7324e0af732 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:07.140879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7779b-efb7-492f-b4ef-8ee661988ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:07.141774 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7779b-efb7-492f-b4ef-8ee661988ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:07.142057 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7779b-efb7-492f-b4ef-8ee661988ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:07.142390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7779b-efb7-492f-b4ef-8ee661988ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:07.143234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7779b-efb7-492f-b4ef-8ee661988ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:07.145686 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7779b-efb7-492f-b4ef-8ee661988ff8 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:07.165898 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 833/1663] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:07.168321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b23408-74b6-4f78-bd56-0d15a9a509ae admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:07.207260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b23408-74b6-4f78-bd56-0d15a9a509ae admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:07.214041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f0938-a005-4cb9-b07b-88593064f54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:07.217032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f0938-a005-4cb9-b07b-88593064f54c admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:07.217466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f0938-a005-4cb9-b07b-88593064f54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:07.217905 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f0938-a005-4cb9-b07b-88593064f54c admin admin] PATH_INFO: `/v3/projects/03391f861ab5454b95e3f149c65651f7/users/465acb54186f4627a44d8fa7379decd6/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:07.265771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657f0938-a005-4cb9-b07b-88593064f54c admin admin] RBAC: Authorizing `identity:create_grant(project_id=03391f861ab5454b95e3f149c65651f7, user_id=465acb54186f4627a44d8fa7379decd6, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:07.277838 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657f0938-a005-4cb9-b07b-88593064f54c admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:07.309592 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 831/1664] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:07 2022] PUT /identity/v3/projects/03391f861ab5454b95e3f149c65651f7/users/465acb54186f4627a44d8fa7379decd6/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:07.315269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7779b-efb7-492f-b4ef-8ee661988ff8 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:07.347447 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7779b-efb7-492f-b4ef-8ee661988ff8 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:07.393892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f0938-a005-4cb9-b07b-88593064f54c admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:07.398148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-657f0938-a005-4cb9-b07b-88593064f54c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:07.584205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f0938-a005-4cb9-b07b-88593064f54c admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:07.595515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f7474-deb8-4515-998d-0078d70489a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:07.597401 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f7474-deb8-4515-998d-0078d70489a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:07.598794 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f7474-deb8-4515-998d-0078d70489a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:07.599553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f7474-deb8-4515-998d-0078d70489a5 admin admin] PATH_INFO: `/v3/projects/cdc3fd148cde4c80a2304f9d9436b420` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:07.599661 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e61e8-6c61-44c4-a953-ee68d74eced7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:07.600423 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e61e8-6c61-44c4-a953-ee68d74eced7 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:07.600859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e61e8-6c61-44c4-a953-ee68d74eced7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:07.601142 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e61e8-6c61-44c4-a953-ee68d74eced7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:07.601980 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e61e8-6c61-44c4-a953-ee68d74eced7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:07.602675 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f7474-deb8-4515-998d-0078d70489a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdc3fd148cde4c80a2304f9d9436b420)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:07.606476 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f7474-deb8-4515-998d-0078d70489a5 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:07.606599 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e61e8-6c61-44c4-a953-ee68d74eced7 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:07.630861 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 834/1665] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:07.634774 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e61e8-6c61-44c4-a953-ee68d74eced7 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:07.677618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e61e8-6c61-44c4-a953-ee68d74eced7 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:07.691510 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ad336-d71e-46e4-92a4-717148bb799f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:07.692796 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ad336-d71e-46e4-92a4-717148bb799f admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:07.694902 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ad336-d71e-46e4-92a4-717148bb799f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:07.695326 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ad336-d71e-46e4-92a4-717148bb799f admin admin] PATH_INFO: `/v3/projects/03391f861ab5454b95e3f149c65651f7/users/465acb54186f4627a44d8fa7379decd6/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:07.701191 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ad336-d71e-46e4-92a4-717148bb799f admin admin] RBAC: Authorizing `identity:create_grant(project_id=03391f861ab5454b95e3f149c65651f7, user_id=465acb54186f4627a44d8fa7379decd6, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:07.704905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ad336-d71e-46e4-92a4-717148bb799f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:07.754568 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 835/1666] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:07 2022] PUT /identity/v3/projects/03391f861ab5454b95e3f149c65651f7/users/465acb54186f4627a44d8fa7379decd6/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:38:07.764247 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d04f7474-deb8-4515-998d-0078d70489a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdc3fd148cde4c80a2304f9d9436b420', 'request_id': 'req-d04f7474-deb8-4515-998d-0078d70489a5'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:07.783011 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d04f7474-deb8-4515-998d-0078d70489a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdc3fd148cde4c80a2304f9d9436b420', 'request_id': 'req-d04f7474-deb8-4515-998d-0078d70489a5'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:07.797447 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d04f7474-deb8-4515-998d-0078d70489a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdc3fd148cde4c80a2304f9d9436b420', 'request_id': 'req-d04f7474-deb8-4515-998d-0078d70489a5'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:07.799604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:07.838631 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d04f7474-deb8-4515-998d-0078d70489a5 admin admin] The token cache is being invalidate because project cdc3fd148cde4c80a2304f9d9436b420 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:07.838631 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d04f7474-deb8-4515-998d-0078d70489a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:07.848291 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d04f7474-deb8-4515-998d-0078d70489a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:07.850380 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 832/1667] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:07 2022] DELETE /identity/v3/projects/cdc3fd148cde4c80a2304f9d9436b420 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:07.855172 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d3534-8202-4269-91c4-186c68a63c63 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:07.855694 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d3534-8202-4269-91c4-186c68a63c63 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:07.856119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d3534-8202-4269-91c4-186c68a63c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:07.928840 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b63d3534-8202-4269-91c4-186c68a63c63 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:07.930660 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b63d3534-8202-4269-91c4-186c68a63c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:08.073549 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cba776-76fa-4bb9-b8f8-69ffc864701d None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:08.074283 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cba776-76fa-4bb9-b8f8-69ffc864701d None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:08.074283 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cba776-76fa-4bb9-b8f8-69ffc864701d None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:08.075818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 833/1668] 10.222.0.42 () {54 vars in 960 bytes} [Mon Jun 6 10:38:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:38:08.083446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3064f-efd9-4578-bbc0-604bc679f69f None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:08.083741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3064f-efd9-4578-bbc0-604bc679f69f None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:08.084074 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3064f-efd9-4578-bbc0-604bc679f69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:08.112806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-3ca3064f-efd9-4578-bbc0-604bc679f69f None None] MFA Rules not processed for user `361be69f1c0a41c18afd206bccf56447`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:08.115239 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-3ca3064f-efd9-4578-bbc0-604bc679f69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:08.177732 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 836/1669] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:08.179606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16704a-4110-44f8-9155-51c1b59200fc None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:08.180147 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16704a-4110-44f8-9155-51c1b59200fc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:08.180535 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16704a-4110-44f8-9155-51c1b59200fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:08.218624 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-6b16704a-4110-44f8-9155-51c1b59200fc None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:08.221204 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6b16704a-4110-44f8-9155-51c1b59200fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:08.230474 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 834/1670] 10.222.0.42 () {58 vars in 1057 bytes} [Mon Jun 6 10:38:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 06 10:38:08.242446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3064f-efd9-4578-bbc0-604bc679f69f None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:08.254008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 837/1671] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:08.258186 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16704a-4110-44f8-9155-51c1b59200fc None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:08.268554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3064f-efd9-4578-bbc0-604bc679f69f None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:08.278514 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16704a-4110-44f8-9155-51c1b59200fc None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:08.282700 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ad19a-eaeb-444e-8b24-34d89e03d3fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361be69f1c0a41c18afd206bccf56447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:08.283609 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ad19a-eaeb-444e-8b24-34d89e03d3fe service placement] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:08.283963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ad19a-eaeb-444e-8b24-34d89e03d3fe service placement] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:08.285040 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ad19a-eaeb-444e-8b24-34d89e03d3fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:08.287526 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a70ad19a-eaeb-444e-8b24-34d89e03d3fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:08.295266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a5bbe-585c-4c11-87af-d475fc4ee61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:08.307668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a5bbe-585c-4c11-87af-d475fc4ee61e admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:08.308133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ad19a-eaeb-444e-8b24-34d89e03d3fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:08.308269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a5bbe-585c-4c11-87af-d475fc4ee61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:08.314197 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a5bbe-585c-4c11-87af-d475fc4ee61e admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:08.315262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a5bbe-585c-4c11-87af-d475fc4ee61e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:38:08.315704 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a5bbe-585c-4c11-87af-d475fc4ee61e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:08.317491 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a5bbe-585c-4c11-87af-d475fc4ee61e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:08.317606 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ad19a-eaeb-444e-8b24-34d89e03d3fe service placement] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:08.323404 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:08.341962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 835/1672] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:08.366871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 838/1673] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:38:08.370619 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ad19a-eaeb-444e-8b24-34d89e03d3fe service placement] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:08.417758 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ad19a-eaeb-444e-8b24-34d89e03d3fe service placement] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:08.444152 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ad34e-2903-4736-bb31-55a4fc8f87f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:08.446456 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ad34e-2903-4736-bb31-55a4fc8f87f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:08.447767 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ad34e-2903-4736-bb31-55a4fc8f87f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:08.448383 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ad34e-2903-4736-bb31-55a4fc8f87f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:08.451885 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ad34e-2903-4736-bb31-55a4fc8f87f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:08.470029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ad34e-2903-4736-bb31-55a4fc8f87f7 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:08.675570 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a5bbe-585c-4c11-87af-d475fc4ee61e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:08.689017 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 836/1674] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:08 2022] POST /identity/v3/projects => generated 387 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:08.694339 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a5bbe-585c-4c11-87af-d475fc4ee61e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:08.927273 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4056d-79c0-413e-8df2-f42321bd720b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:08.928144 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4056d-79c0-413e-8df2-f42321bd720b admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:08.928563 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4056d-79c0-413e-8df2-f42321bd720b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:08.928973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4056d-79c0-413e-8df2-f42321bd720b admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:08.929822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4056d-79c0-413e-8df2-f42321bd720b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:08.932009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4056d-79c0-413e-8df2-f42321bd720b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:08.992943 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 839/1675] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:08 2022] POST /identity/v3/users => generated 294 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:38:08.995198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ad34e-2903-4736-bb31-55a4fc8f87f7 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:09.041807 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ad34e-2903-4736-bb31-55a4fc8f87f7 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:09.053819 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223ecb6-1fb1-40f9-af52-f30e0f6c8023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:09.056738 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223ecb6-1fb1-40f9-af52-f30e0f6c8023 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:09.057166 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223ecb6-1fb1-40f9-af52-f30e0f6c8023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:09.057726 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223ecb6-1fb1-40f9-af52-f30e0f6c8023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:09.060994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223ecb6-1fb1-40f9-af52-f30e0f6c8023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:09.062700 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223ecb6-1fb1-40f9-af52-f30e0f6c8023 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:09.148045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 837/1676] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:09.151718 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4056d-79c0-413e-8df2-f42321bd720b admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:09.175166 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4056d-79c0-413e-8df2-f42321bd720b admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:09.186819 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0ff1b-d564-4edb-9d58-63cbaf0bcb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:09.187673 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0ff1b-d564-4edb-9d58-63cbaf0bcb63 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:09.187932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0ff1b-d564-4edb-9d58-63cbaf0bcb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:09.188213 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0ff1b-d564-4edb-9d58-63cbaf0bcb63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:09.188995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c0ff1b-d564-4edb-9d58-63cbaf0bcb63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:09.199938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c0ff1b-d564-4edb-9d58-63cbaf0bcb63 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:09.227122 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 840/1677] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:09.232216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223ecb6-1fb1-40f9-af52-f30e0f6c8023 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:09.282530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223ecb6-1fb1-40f9-af52-f30e0f6c8023 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:09.313566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a240153-d3bf-4d0a-a125-4581a1587583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:09.315051 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a240153-d3bf-4d0a-a125-4581a1587583 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:09.315635 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a240153-d3bf-4d0a-a125-4581a1587583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:09.316358 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a240153-d3bf-4d0a-a125-4581a1587583 admin admin] PATH_INFO: `/v3/projects/2b39d39669d94efcac1d3504acdfd22c/users/64364f46853b4b76a1dde5d9f55d582b/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:09.354963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a240153-d3bf-4d0a-a125-4581a1587583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b39d39669d94efcac1d3504acdfd22c, user_id=64364f46853b4b76a1dde5d9f55d582b, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:09.358421 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a240153-d3bf-4d0a-a125-4581a1587583 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:09.389539 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 838/1678] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:09 2022] PUT /identity/v3/projects/2b39d39669d94efcac1d3504acdfd22c/users/64364f46853b4b76a1dde5d9f55d582b/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:09.409254 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba07822-a90d-4f26-92a8-32a6912b3fe9 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:09.409606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba07822-a90d-4f26-92a8-32a6912b3fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:09.409967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba07822-a90d-4f26-92a8-32a6912b3fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:09.479136 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3ba07822-a90d-4f26-92a8-32a6912b3fe9 None None] MFA Rules not processed for user `64364f46853b4b76a1dde5d9f55d582b`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:09.483196 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3ba07822-a90d-4f26-92a8-32a6912b3fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:09.501062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a240153-d3bf-4d0a-a125-4581a1587583 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:09.503701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-7a240153-d3bf-4d0a-a125-4581a1587583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:09.649289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 841/1679] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:09.654060 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba07822-a90d-4f26-92a8-32a6912b3fe9 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:09.666836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba07822-a90d-4f26-92a8-32a6912b3fe9 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:09.679500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a240153-d3bf-4d0a-a125-4581a1587583 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:09.682478 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492cec5-f63d-40e8-b025-42e7819088dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:09.683433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492cec5-f63d-40e8-b025-42e7819088dc admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:09.684117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492cec5-f63d-40e8-b025-42e7819088dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:09.684788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492cec5-f63d-40e8-b025-42e7819088dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:09.685878 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492cec5-f63d-40e8-b025-42e7819088dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:09.688981 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492cec5-f63d-40e8-b025-42e7819088dc admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:09.692743 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad56140-08e4-483e-8e50-b83dba3aeb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:09.693645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad56140-08e4-483e-8e50-b83dba3aeb38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:09.694111 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad56140-08e4-483e-8e50-b83dba3aeb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:09.694443 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad56140-08e4-483e-8e50-b83dba3aeb38 admin admin] PATH_INFO: `/v3/projects/dda2ef0ed9234862bcf3af1f94b1525d` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:09.696915 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad56140-08e4-483e-8e50-b83dba3aeb38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dda2ef0ed9234862bcf3af1f94b1525d)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:09.703633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad56140-08e4-483e-8e50-b83dba3aeb38 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:09.761370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 842/1680] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:09 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:09.766612 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492cec5-f63d-40e8-b025-42e7819088dc admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:09.778586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:09.828605 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-bad56140-08e4-483e-8e50-b83dba3aeb38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dda2ef0ed9234862bcf3af1f94b1525d', 'request_id': 'req-bad56140-08e4-483e-8e50-b83dba3aeb38'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:09.836588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-bad56140-08e4-483e-8e50-b83dba3aeb38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dda2ef0ed9234862bcf3af1f94b1525d', 'request_id': 'req-bad56140-08e4-483e-8e50-b83dba3aeb38'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:09.860009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-bad56140-08e4-483e-8e50-b83dba3aeb38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dda2ef0ed9234862bcf3af1f94b1525d', 'request_id': 'req-bad56140-08e4-483e-8e50-b83dba3aeb38'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:09.879689 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-bad56140-08e4-483e-8e50-b83dba3aeb38 admin admin] The token cache is being invalidate because project dda2ef0ed9234862bcf3af1f94b1525d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:09.880060 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-bad56140-08e4-483e-8e50-b83dba3aeb38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:09.880708 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-bad56140-08e4-483e-8e50-b83dba3aeb38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:09.885793 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 839/1681] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:09 2022] DELETE /identity/v3/projects/dda2ef0ed9234862bcf3af1f94b1525d => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:09.887627 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35691c9e-bdc2-4c85-bc6c-0eefce7b8120 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:09.888133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35691c9e-bdc2-4c85-bc6c-0eefce7b8120 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:09.890756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35691c9e-bdc2-4c85-bc6c-0eefce7b8120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:09.904381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:09.951215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-35691c9e-bdc2-4c85-bc6c-0eefce7b8120 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:09.959986 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492cec5-f63d-40e8-b025-42e7819088dc admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:09.961769 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-35691c9e-bdc2-4c85-bc6c-0eefce7b8120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:10.122993 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878b022-57d2-42b3-8e70-4c78459f66a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:10.124826 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878b022-57d2-42b3-8e70-4c78459f66a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:10.125339 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878b022-57d2-42b3-8e70-4c78459f66a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:10.125826 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878b022-57d2-42b3-8e70-4c78459f66a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:10.127295 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e878b022-57d2-42b3-8e70-4c78459f66a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:10.134734 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e878b022-57d2-42b3-8e70-4c78459f66a9 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:10.170316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 840/1682] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:38:10.174920 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 843/1683] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:09 2022] POST /identity/v3/users => generated 300 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:10.179074 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc77a5-9ffc-4d06-889e-cb8ef9393d39 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:10.179074 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc77a5-9ffc-4d06-889e-cb8ef9393d39 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:10.179247 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc77a5-9ffc-4d06-889e-cb8ef9393d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:10.186382 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878b022-57d2-42b3-8e70-4c78459f66a9 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:10.193357 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e878b022-57d2-42b3-8e70-4c78459f66a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:10.230636 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878b022-57d2-42b3-8e70-4c78459f66a9 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:10.235916 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-e9dc77a5-9ffc-4d06-889e-cb8ef9393d39 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:10.237322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-e9dc77a5-9ffc-4d06-889e-cb8ef9393d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:10.263823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5b692-d4e3-4a21-9297-dee61263a2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:10.263823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5b692-d4e3-4a21-9297-dee61263a2ef admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:10.263823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5b692-d4e3-4a21-9297-dee61263a2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:10.263823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5b692-d4e3-4a21-9297-dee61263a2ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:10.264263 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5b692-d4e3-4a21-9297-dee61263a2ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:10.264447 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5b692-d4e3-4a21-9297-dee61263a2ef admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:10.268273 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 841/1684] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:10.272694 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc77a5-9ffc-4d06-889e-cb8ef9393d39 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:10.296587 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc77a5-9ffc-4d06-889e-cb8ef9393d39 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:10.300399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 844/1685] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:10.301395 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5b692-d4e3-4a21-9297-dee61263a2ef admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:10.315876 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d2d66-9c2c-41cc-8b87-73185fabaadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:10.316359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d2d66-9c2c-41cc-8b87-73185fabaadc admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:10.316648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d2d66-9c2c-41cc-8b87-73185fabaadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:10.316931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d2d66-9c2c-41cc-8b87-73185fabaadc admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:10.317683 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d2d66-9c2c-41cc-8b87-73185fabaadc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:38:10.322782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d2d66-9c2c-41cc-8b87-73185fabaadc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:10.329252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d2d66-9c2c-41cc-8b87-73185fabaadc admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:10.332357 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5b692-d4e3-4a21-9297-dee61263a2ef admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:10.350681 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1697a7-0169-454f-9f8e-0f6ec6195ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:10.351742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1697a7-0169-454f-9f8e-0f6ec6195ad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:10.352061 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1697a7-0169-454f-9f8e-0f6ec6195ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:10.352342 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1697a7-0169-454f-9f8e-0f6ec6195ad9 admin admin] PATH_INFO: `/v3/projects/73f77838dcb2424081c1f0d20647a0ac/users/70f1bdde9294473d83233ace443ce799/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:10.376569 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 842/1686] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:38:10.381637 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d2d66-9c2c-41cc-8b87-73185fabaadc admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:10.402760 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1697a7-0169-454f-9f8e-0f6ec6195ad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73f77838dcb2424081c1f0d20647a0ac, user_id=70f1bdde9294473d83233ace443ce799, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:10.405934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1697a7-0169-454f-9f8e-0f6ec6195ad9 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:10.409345 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d2d66-9c2c-41cc-8b87-73185fabaadc admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:10.415384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4ee78-04d2-4313-ad7a-20ac58eae17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:10.416532 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4ee78-04d2-4313-ad7a-20ac58eae17b admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:10.416823 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4ee78-04d2-4313-ad7a-20ac58eae17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:10.417107 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4ee78-04d2-4313-ad7a-20ac58eae17b admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:10.417817 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4ee78-04d2-4313-ad7a-20ac58eae17b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:10.419768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4ee78-04d2-4313-ad7a-20ac58eae17b admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:10.445698 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 845/1687] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:10 2022] PUT /identity/v3/projects/73f77838dcb2424081c1f0d20647a0ac/users/70f1bdde9294473d83233ace443ce799/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:10.449707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1697a7-0169-454f-9f8e-0f6ec6195ad9 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:10.530817 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 843/1688] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:10 2022] POST /identity/v3/projects => generated 387 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:10.534987 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4ee78-04d2-4313-ad7a-20ac58eae17b admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:10.653057 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1697a7-0169-454f-9f8e-0f6ec6195ad9 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:10.706794 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4ee78-04d2-4313-ad7a-20ac58eae17b admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:10.713960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1601a10-5104-4c4c-a962-d2163eda65d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:10.714797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1601a10-5104-4c4c-a962-d2163eda65d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:10.714967 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1601a10-5104-4c4c-a962-d2163eda65d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:10.715214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1601a10-5104-4c4c-a962-d2163eda65d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:10.716211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1601a10-5104-4c4c-a962-d2163eda65d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:10.718021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1601a10-5104-4c4c-a962-d2163eda65d4 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:10.772896 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:10.786349 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 844/1689] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:10 2022] POST /identity/v3/users => generated 294 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:10.787953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1601a10-5104-4c4c-a962-d2163eda65d4 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:10.792586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:10.820426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff922ec-0cda-4041-928c-6e1d55e242a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:10.821137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff922ec-0cda-4041-928c-6e1d55e242a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:10.821494 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff922ec-0cda-4041-928c-6e1d55e242a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:10.821844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff922ec-0cda-4041-928c-6e1d55e242a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:10.822122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1601a10-5104-4c4c-a962-d2163eda65d4 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:10.822721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff922ec-0cda-4041-928c-6e1d55e242a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:10.824883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff922ec-0cda-4041-928c-6e1d55e242a2 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:10.829105 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e37ce-a24e-4abd-ae58-8bd09e5d5c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:10.829636 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e37ce-a24e-4abd-ae58-8bd09e5d5c67 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:10.829922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e37ce-a24e-4abd-ae58-8bd09e5d5c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:10.830207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e37ce-a24e-4abd-ae58-8bd09e5d5c67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:10.830996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e37ce-a24e-4abd-ae58-8bd09e5d5c67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:10.837034 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e37ce-a24e-4abd-ae58-8bd09e5d5c67 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:10.857257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 846/1690] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:10.863550 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 845/1691] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:10.864540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff922ec-0cda-4041-928c-6e1d55e242a2 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:10.864649 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e37ce-a24e-4abd-ae58-8bd09e5d5c67 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:10.887121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e37ce-a24e-4abd-ae58-8bd09e5d5c67 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:10.892592 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2a0ab-3cfc-48bf-a7ef-9ca713641d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:10.893277 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2a0ab-3cfc-48bf-a7ef-9ca713641d1b admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:10.893561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2a0ab-3cfc-48bf-a7ef-9ca713641d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:10.893837 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2a0ab-3cfc-48bf-a7ef-9ca713641d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:10.894568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2a0ab-3cfc-48bf-a7ef-9ca713641d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:10.896466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2a0ab-3cfc-48bf-a7ef-9ca713641d1b admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:10.918412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff922ec-0cda-4041-928c-6e1d55e242a2 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:10.918724 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 846/1692] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:10.923722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2a0ab-3cfc-48bf-a7ef-9ca713641d1b admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:10.937597 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2a0ab-3cfc-48bf-a7ef-9ca713641d1b admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:10.942480 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d5446-4a3c-4db6-b751-7459c17e9de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:10.942925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07715e-f374-4cc2-94d0-3f9a79336d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:10.943861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07715e-f374-4cc2-94d0-3f9a79336d05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:10.944150 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07715e-f374-4cc2-94d0-3f9a79336d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:10.944422 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07715e-f374-4cc2-94d0-3f9a79336d05 admin admin] PATH_INFO: `/v3/projects/90787cbf98cc4029858f9baf365eb4d5/users/02e3ca5d52a84e0aa4b22a20e7621fc9/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:10.947583 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d5446-4a3c-4db6-b751-7459c17e9de6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:10.950288 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d5446-4a3c-4db6-b751-7459c17e9de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:10.950582 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d5446-4a3c-4db6-b751-7459c17e9de6 admin admin] PATH_INFO: `/v3/projects/73f77838dcb2424081c1f0d20647a0ac/users/70f1bdde9294473d83233ace443ce799/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:10.958392 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d5446-4a3c-4db6-b751-7459c17e9de6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73f77838dcb2424081c1f0d20647a0ac, user_id=70f1bdde9294473d83233ace443ce799, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:10.961727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d5446-4a3c-4db6-b751-7459c17e9de6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:10.977188 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a07715e-f374-4cc2-94d0-3f9a79336d05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90787cbf98cc4029858f9baf365eb4d5, user_id=02e3ca5d52a84e0aa4b22a20e7621fc9, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:10.981325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a07715e-f374-4cc2-94d0-3f9a79336d05 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:10.993834 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 847/1693] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:10 2022] PUT /identity/v3/projects/73f77838dcb2424081c1f0d20647a0ac/users/70f1bdde9294473d83233ace443ce799/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:11.005879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c810e9a-61ff-4a59-ba1b-cdf2b08ae405 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:11.006114 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c810e9a-61ff-4a59-ba1b-cdf2b08ae405 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:11.006592 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c810e9a-61ff-4a59-ba1b-cdf2b08ae405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:11.021588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 847/1694] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:10 2022] PUT /identity/v3/projects/90787cbf98cc4029858f9baf365eb4d5/users/02e3ca5d52a84e0aa4b22a20e7621fc9/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:38:11.027192 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10533a4-2ab9-412d-bb60-e88ebe0bf727 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:11.027450 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10533a4-2ab9-412d-bb60-e88ebe0bf727 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:11.027739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10533a4-2ab9-412d-bb60-e88ebe0bf727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:11.109554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-e10533a4-2ab9-412d-bb60-e88ebe0bf727 None None] MFA Rules not processed for user `02e3ca5d52a84e0aa4b22a20e7621fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:11.121256 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-0c810e9a-61ff-4a59-ba1b-cdf2b08ae405 None None] MFA Rules not processed for user `70f1bdde9294473d83233ace443ce799`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:11.122704 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-e10533a4-2ab9-412d-bb60-e88ebe0bf727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:11.126294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0c810e9a-61ff-4a59-ba1b-cdf2b08ae405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:11.241031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 848/1695] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:11.274583 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c810e9a-61ff-4a59-ba1b-cdf2b08ae405 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:11.279039 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0c810e9a-61ff-4a59-ba1b-cdf2b08ae405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:11.286792 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 848/1696] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:11.287718 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10533a4-2ab9-412d-bb60-e88ebe0bf727 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:11.479412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c810e9a-61ff-4a59-ba1b-cdf2b08ae405 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:11.484836 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10533a4-2ab9-412d-bb60-e88ebe0bf727 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:11.488026 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8a1e1-b19d-4907-9375-440f63836ee9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:11.488782 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8a1e1-b19d-4907-9375-440f63836ee9 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:11.489282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8a1e1-b19d-4907-9375-440f63836ee9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:11.489769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8a1e1-b19d-4907-9375-440f63836ee9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:11.496027 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d190817-4839-4eb9-a434-21336c1d2ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:11.497167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d190817-4839-4eb9-a434-21336c1d2ed9 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:11.497862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d190817-4839-4eb9-a434-21336c1d2ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:11.498427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d190817-4839-4eb9-a434-21336c1d2ed9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:11.500641 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d190817-4839-4eb9-a434-21336c1d2ed9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:11.504442 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d190817-4839-4eb9-a434-21336c1d2ed9 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:11.515576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8a1e1-b19d-4907-9375-440f63836ee9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:11.518823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8a1e1-b19d-4907-9375-440f63836ee9 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:11.534336 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 849/1697] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:38:11.585977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 849/1698] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:11 2022] POST /identity/v3/projects => generated 387 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:11.588427 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8a1e1-b19d-4907-9375-440f63836ee9 service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:11.618228 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8a1e1-b19d-4907-9375-440f63836ee9 service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:11.724032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c58c2-7866-4663-91cb-be7bf8fdc6b8 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:11.724568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c58c2-7866-4663-91cb-be7bf8fdc6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:11.724807 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c58c2-7866-4663-91cb-be7bf8fdc6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:11.762540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990b030-de7f-4810-ac83-7ae039935a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:11.764418 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990b030-de7f-4810-ac83-7ae039935a24 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:11.766732 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990b030-de7f-4810-ac83-7ae039935a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:11.767125 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990b030-de7f-4810-ac83-7ae039935a24 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:11.767976 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990b030-de7f-4810-ac83-7ae039935a24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:11.775544 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990b030-de7f-4810-ac83-7ae039935a24 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:11.794516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:11.818764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 850/1699] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:11 2022] POST /identity/v3/users => generated 300 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:11.822913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990b030-de7f-4810-ac83-7ae039935a24 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:11.841249 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-091c58c2-7866-4663-91cb-be7bf8fdc6b8 None None] MFA Rules not processed for user `465acb54186f4627a44d8fa7379decd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:11.846209 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990b030-de7f-4810-ac83-7ae039935a24 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:11.854765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-091c58c2-7866-4663-91cb-be7bf8fdc6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:11.878674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff733b-a656-4d47-bdef-2a45b8921557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:11.879387 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff733b-a656-4d47-bdef-2a45b8921557 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:11.879670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff733b-a656-4d47-bdef-2a45b8921557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:11.880012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff733b-a656-4d47-bdef-2a45b8921557 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:11.887295 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff733b-a656-4d47-bdef-2a45b8921557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:11.898716 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff733b-a656-4d47-bdef-2a45b8921557 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:11.926789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:11.944999 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 851/1700] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:38:11.947920 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff733b-a656-4d47-bdef-2a45b8921557 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:11.974113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff733b-a656-4d47-bdef-2a45b8921557 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:11.985053 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d330f-ff29-494e-8d98-3b142cc35ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:11.985976 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d330f-ff29-494e-8d98-3b142cc35ce4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:11.986264 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d330f-ff29-494e-8d98-3b142cc35ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:11.986540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d330f-ff29-494e-8d98-3b142cc35ce4 admin admin] PATH_INFO: `/v3/projects/ed9758dd444c45db9d78dbd04810507c/users/3b978f134fe04792978a8790365329d1/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:12.040843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d330f-ff29-494e-8d98-3b142cc35ce4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed9758dd444c45db9d78dbd04810507c, user_id=3b978f134fe04792978a8790365329d1, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:12.049256 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d330f-ff29-494e-8d98-3b142cc35ce4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:12.094678 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 852/1701] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:11 2022] PUT /identity/v3/projects/ed9758dd444c45db9d78dbd04810507c/users/3b978f134fe04792978a8790365329d1/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:38:12.095627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d330f-ff29-494e-8d98-3b142cc35ce4 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:12.131827 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 850/1702] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:11 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:38:12.249210 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d330f-ff29-494e-8d98-3b142cc35ce4 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:12.262071 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01bde6-7ffd-4ab2-82a6-58466250fbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:12.262768 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01bde6-7ffd-4ab2-82a6-58466250fbee admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:12.263053 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01bde6-7ffd-4ab2-82a6-58466250fbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:12.263331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01bde6-7ffd-4ab2-82a6-58466250fbee admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:12.264098 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01bde6-7ffd-4ab2-82a6-58466250fbee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:12.266491 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01bde6-7ffd-4ab2-82a6-58466250fbee admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:12.288588 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 853/1703] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:12.292875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c58c2-7866-4663-91cb-be7bf8fdc6b8 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:12.338034 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c58c2-7866-4663-91cb-be7bf8fdc6b8 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:12.361468 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc0060-6573-40ee-8db7-30bb89596b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:12.362761 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc0060-6573-40ee-8db7-30bb89596b49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:12.363529 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc0060-6573-40ee-8db7-30bb89596b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:12.364003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc0060-6573-40ee-8db7-30bb89596b49 admin admin] PATH_INFO: `/v3/projects/ed9758dd444c45db9d78dbd04810507c/users/3b978f134fe04792978a8790365329d1/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:12.382299 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc0060-6573-40ee-8db7-30bb89596b49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed9758dd444c45db9d78dbd04810507c, user_id=3b978f134fe04792978a8790365329d1, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:12.385320 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc0060-6573-40ee-8db7-30bb89596b49 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:12.433996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 851/1704] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:12 2022] PUT /identity/v3/projects/ed9758dd444c45db9d78dbd04810507c/users/3b978f134fe04792978a8790365329d1/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:12.448775 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c95ea-f506-4c57-9173-94312cfdba76 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:12.449064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c95ea-f506-4c57-9173-94312cfdba76 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:12.449340 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c95ea-f506-4c57-9173-94312cfdba76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:12.451233 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2c680-dfb1-4b3a-8550-c71caebbb356 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:12.451739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2c680-dfb1-4b3a-8550-c71caebbb356 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:12.452386 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2c680-dfb1-4b3a-8550-c71caebbb356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:12.483421 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-2ef2c680-dfb1-4b3a-8550-c71caebbb356 None None] MFA Rules not processed for user `e9e358afe0974c0c8bd53104ebbff445`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:12.485302 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-2ef2c680-dfb1-4b3a-8550-c71caebbb356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:12.500364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-a86c95ea-f506-4c57-9173-94312cfdba76 None None] MFA Rules not processed for user `3b978f134fe04792978a8790365329d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:12.514900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a86c95ea-f506-4c57-9173-94312cfdba76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:12.603685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 852/1705] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jun 6 10:38:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:12.767238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 854/1706] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:12.781296 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2c680-dfb1-4b3a-8550-c71caebbb356 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:12.820500 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c95ea-f506-4c57-9173-94312cfdba76 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:12.856658 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2c680-dfb1-4b3a-8550-c71caebbb356 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:12.861945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b190e-5369-4b27-929d-f4d59465debd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:12.862905 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b190e-5369-4b27-929d-f4d59465debd service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:12.863402 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b190e-5369-4b27-929d-f4d59465debd service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:12.864137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b190e-5369-4b27-929d-f4d59465debd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:12.878870 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b190e-5369-4b27-929d-f4d59465debd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:12.880802 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b190e-5369-4b27-929d-f4d59465debd service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:12.894645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 853/1707] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:12.918460 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:12.943359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c95ea-f506-4c57-9173-94312cfdba76 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:12.949976 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf335a14-2ba9-40a3-8610-47780e24b27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:12.951673 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf335a14-2ba9-40a3-8610-47780e24b27a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:12.953356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf335a14-2ba9-40a3-8610-47780e24b27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:12.953768 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf335a14-2ba9-40a3-8610-47780e24b27a admin admin] PATH_INFO: `/v3/users/64364f46853b4b76a1dde5d9f55d582b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:12.962737 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b190e-5369-4b27-929d-f4d59465debd service cinder] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:12.980892 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf335a14-2ba9-40a3-8610-47780e24b27a admin admin] RBAC: Authorizing `identity:delete_user(user_id=64364f46853b4b76a1dde5d9f55d582b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:12.982860 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf335a14-2ba9-40a3-8610-47780e24b27a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:12.986293 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b190e-5369-4b27-929d-f4d59465debd service cinder] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:13.000258 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c98cec-6b2e-4a6f-bdc8-971588cd3927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:13.001316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c98cec-6b2e-4a6f-bdc8-971588cd3927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:13.001604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c98cec-6b2e-4a6f-bdc8-971588cd3927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:13.001888 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c98cec-6b2e-4a6f-bdc8-971588cd3927 admin admin] PATH_INFO: `/v3/users/02e3ca5d52a84e0aa4b22a20e7621fc9` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:13.003347 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c98cec-6b2e-4a6f-bdc8-971588cd3927 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02e3ca5d52a84e0aa4b22a20e7621fc9)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:13.007927 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c98cec-6b2e-4a6f-bdc8-971588cd3927 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:13.119152 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-cf335a14-2ba9-40a3-8610-47780e24b27a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64364f46853b4b76a1dde5d9f55d582b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:13.138193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:13.144767 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-cf335a14-2ba9-40a3-8610-47780e24b27a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64364f46853b4b76a1dde5d9f55d582b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:13.146216 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-cf335a14-2ba9-40a3-8610-47780e24b27a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64364f46853b4b76a1dde5d9f55d582b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:13.148279 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-cf335a14-2ba9-40a3-8610-47780e24b27a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64364f46853b4b76a1dde5d9f55d582b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:13.154040 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-93c98cec-6b2e-4a6f-bdc8-971588cd3927 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02e3ca5d52a84e0aa4b22a20e7621fc9'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:13.166370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-cf335a14-2ba9-40a3-8610-47780e24b27a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64364f46853b4b76a1dde5d9f55d582b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:13.175536 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-93c98cec-6b2e-4a6f-bdc8-971588cd3927 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02e3ca5d52a84e0aa4b22a20e7621fc9'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:13.185758 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-93c98cec-6b2e-4a6f-bdc8-971588cd3927 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02e3ca5d52a84e0aa4b22a20e7621fc9'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:13.187495 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-93c98cec-6b2e-4a6f-bdc8-971588cd3927 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02e3ca5d52a84e0aa4b22a20e7621fc9'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:13.195519 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 855/1708] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:12 2022] DELETE /identity/v3/users/64364f46853b4b76a1dde5d9f55d582b => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:13.196396 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf335a14-2ba9-40a3-8610-47780e24b27a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:13.200454 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-cf335a14-2ba9-40a3-8610-47780e24b27a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:13.207891 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-93c98cec-6b2e-4a6f-bdc8-971588cd3927 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02e3ca5d52a84e0aa4b22a20e7621fc9'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:13.218762 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 854/1709] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:12 2022] DELETE /identity/v3/users/02e3ca5d52a84e0aa4b22a20e7621fc9 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:13.221048 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec46e4-fbef-49bd-be8a-5cc091f4ce24 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:13.221536 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec46e4-fbef-49bd-be8a-5cc091f4ce24 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:13.222086 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec46e4-fbef-49bd-be8a-5cc091f4ce24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:13.256871 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-35ec46e4-fbef-49bd-be8a-5cc091f4ce24 None None] MFA Rules not processed for user `f6b7d82c999646b6b6b96d565f0e3f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:13.259202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-35ec46e4-fbef-49bd-be8a-5cc091f4ce24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:13.361376 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 855/1710] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:13.362372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ec46e4-fbef-49bd-be8a-5cc091f4ce24 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:13.364701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-35ec46e4-fbef-49bd-be8a-5cc091f4ce24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:13.377547 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf335a14-2ba9-40a3-8610-47780e24b27a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:13.489727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:13.503861 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8384b-eeaf-496c-8492-ae9428b43846 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fede9e52ad274285b09d748a684ede38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:13.504574 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8384b-eeaf-496c-8492-ae9428b43846 service swift] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:13.504878 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8384b-eeaf-496c-8492-ae9428b43846 service swift] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:13.505176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8384b-eeaf-496c-8492-ae9428b43846 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:13.509984 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fde8384b-eeaf-496c-8492-ae9428b43846 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:13.518304 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ec46e4-fbef-49bd-be8a-5cc091f4ce24 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:13.525763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb27a1-4194-435b-8df5-c2908d1b7423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:13.529281 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb27a1-4194-435b-8df5-c2908d1b7423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:13.529571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb27a1-4194-435b-8df5-c2908d1b7423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:13.529907 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb27a1-4194-435b-8df5-c2908d1b7423 admin admin] PATH_INFO: `/v3/users/749aa2ca183140e2a86d6be9aac6b4a4` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:13.532808 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb27a1-4194-435b-8df5-c2908d1b7423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=749aa2ca183140e2a86d6be9aac6b4a4)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:13.535865 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb27a1-4194-435b-8df5-c2908d1b7423 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:13.664793 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-68bb27a1-4194-435b-8df5-c2908d1b7423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '749aa2ca183140e2a86d6be9aac6b4a4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:13.671452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:13.686983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8384b-eeaf-496c-8492-ae9428b43846 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:13.689062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-68bb27a1-4194-435b-8df5-c2908d1b7423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '749aa2ca183140e2a86d6be9aac6b4a4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:13.691152 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8384b-eeaf-496c-8492-ae9428b43846 service swift] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:13.711218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-68bb27a1-4194-435b-8df5-c2908d1b7423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '749aa2ca183140e2a86d6be9aac6b4a4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:13.711961 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-68bb27a1-4194-435b-8df5-c2908d1b7423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '749aa2ca183140e2a86d6be9aac6b4a4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:13.722564 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 856/1711] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jun 6 10:38:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 10:38:13.726447 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9fa4d-61f8-44a2-a7f1-19d560cb692e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:13.726751 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9fa4d-61f8-44a2-a7f1-19d560cb692e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:13.728626 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9fa4d-61f8-44a2-a7f1-19d560cb692e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:13.740655 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-68bb27a1-4194-435b-8df5-c2908d1b7423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '749aa2ca183140e2a86d6be9aac6b4a4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:13.772426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 856/1712] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:13 2022] DELETE /identity/v3/users/749aa2ca183140e2a86d6be9aac6b4a4 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:13.777523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9a299-747d-4742-9f4f-4a046dcf8a9f None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:13.777939 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9a299-747d-4742-9f4f-4a046dcf8a9f None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:13.778395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9a299-747d-4742-9f4f-4a046dcf8a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:13.790693 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-10a9fa4d-61f8-44a2-a7f1-19d560cb692e None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:13.796872 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-10a9fa4d-61f8-44a2-a7f1-19d560cb692e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:13.834455 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-a7f9a299-747d-4742-9f4f-4a046dcf8a9f None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:13.837285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a7f9a299-747d-4742-9f4f-4a046dcf8a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:14.003499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 857/1713] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:14.009194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9fa4d-61f8-44a2-a7f1-19d560cb692e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:14.021352 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-10a9fa4d-61f8-44a2-a7f1-19d560cb692e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:14.073452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 857/1714] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:14.074953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643bc89-d626-43e9-ba49-c9fc6016a2f7 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:14.075342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643bc89-d626-43e9-ba49-c9fc6016a2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:14.075748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643bc89-d626-43e9-ba49-c9fc6016a2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:14.115773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-a643bc89-d626-43e9-ba49-c9fc6016a2f7 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:14.123062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a643bc89-d626-43e9-ba49-c9fc6016a2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:14.140315 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 858/1715] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:14.144541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643bc89-d626-43e9-ba49-c9fc6016a2f7 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:14.154855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a643bc89-d626-43e9-ba49-c9fc6016a2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:14.246840 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9fa4d-61f8-44a2-a7f1-19d560cb692e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:14.276709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643bc89-d626-43e9-ba49-c9fc6016a2f7 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:14.282569 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5131165-385e-43db-8a89-bb80ea345444 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:14.283696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5131165-385e-43db-8a89-bb80ea345444 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:14.284261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5131165-385e-43db-8a89-bb80ea345444 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:14.284745 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5131165-385e-43db-8a89-bb80ea345444 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:14.289574 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bdcb09-ee77-45aa-9f8f-9e0f45886519 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:14.290333 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdcb09-ee77-45aa-9f8f-9e0f45886519 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:14.290635 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdcb09-ee77-45aa-9f8f-9e0f45886519 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:14.290919 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdcb09-ee77-45aa-9f8f-9e0f45886519 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:14.300883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-65bdcb09-ee77-45aa-9f8f-9e0f45886519 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:14.310770 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5131165-385e-43db-8a89-bb80ea345444 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:14.312914 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5131165-385e-43db-8a89-bb80ea345444 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:14.337951 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 859/1716] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:14.339194 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5131165-385e-43db-8a89-bb80ea345444 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:14.357254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5131165-385e-43db-8a89-bb80ea345444 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:14.372265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd447cd-ff6c-440e-a65e-3360e6ac679b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:14.373323 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd447cd-ff6c-440e-a65e-3360e6ac679b service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:14.373818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd447cd-ff6c-440e-a65e-3360e6ac679b service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:14.375478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd447cd-ff6c-440e-a65e-3360e6ac679b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:14.419859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd447cd-ff6c-440e-a65e-3360e6ac679b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:14.423980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd447cd-ff6c-440e-a65e-3360e6ac679b service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:14.433557 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:14.462165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 860/1717] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:14.463424 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd447cd-ff6c-440e-a65e-3360e6ac679b service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:14.495482 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd447cd-ff6c-440e-a65e-3360e6ac679b service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:14.495831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bdcb09-ee77-45aa-9f8f-9e0f45886519 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:14.501867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bdcb09-ee77-45aa-9f8f-9e0f45886519 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:14.506980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a227f2f0-0b1e-4590-a8fc-fcd8bf5a75ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:14.507875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227f2f0-0b1e-4590-a8fc-fcd8bf5a75ea service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:14.508395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227f2f0-0b1e-4590-a8fc-fcd8bf5a75ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:14.508956 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227f2f0-0b1e-4590-a8fc-fcd8bf5a75ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:14.540674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 858/1718] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:14.542552 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a227f2f0-0b1e-4590-a8fc-fcd8bf5a75ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:14.544316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a227f2f0-0b1e-4590-a8fc-fcd8bf5a75ea service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:14.570381 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 861/1719] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:15.344464 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d9a04-b5f8-40db-98d8-c01cd4176589 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:15.345353 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d9a04-b5f8-40db-98d8-c01cd4176589 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:15.345789 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d9a04-b5f8-40db-98d8-c01cd4176589 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:15.348015 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 859/1720] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jun 6 10:38:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 10:38:15.384625 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d1df3-6252-440d-afd4-96f439527cab None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:15.387103 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d1df3-6252-440d-afd4-96f439527cab None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:15.388477 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d1df3-6252-440d-afd4-96f439527cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:15.447595 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-591d1df3-6252-440d-afd4-96f439527cab None None] MFA Rules not processed for user `361be69f1c0a41c18afd206bccf56447`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:15.450921 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-591d1df3-6252-440d-afd4-96f439527cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:15.567887 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:15.668516 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 862/1721] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jun 6 10:38:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:15.876220 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66edb7-e658-4b3c-93df-9d21dc1e19a6 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:15.877146 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66edb7-e658-4b3c-93df-9d21dc1e19a6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:15.877146 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66edb7-e658-4b3c-93df-9d21dc1e19a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:15.929027 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-bd66edb7-e658-4b3c-93df-9d21dc1e19a6 None None] MFA Rules not processed for user `881069f9b40a485dac2dda684dd6cd3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:15.930340 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bd66edb7-e658-4b3c-93df-9d21dc1e19a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:15.961493 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 860/1722] 10.222.0.46 () {58 vars in 1015 bytes} [Mon Jun 6 10:38:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:16.011297 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d1df3-6252-440d-afd4-96f439527cab None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:16.012688 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-591d1df3-6252-440d-afd4-96f439527cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:16.037089 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d1df3-6252-440d-afd4-96f439527cab None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:16.048917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a2a3a-d83c-4131-bed6-4182546d15b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:16.049509 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a2a3a-d83c-4131-bed6-4182546d15b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:16.049801 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a2a3a-d83c-4131-bed6-4182546d15b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:16.050218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a2a3a-d83c-4131-bed6-4182546d15b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:16.073582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a2a3a-d83c-4131-bed6-4182546d15b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:16.075895 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a2a3a-d83c-4131-bed6-4182546d15b9 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:16.087043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 863/1723] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:16.254739 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66edb7-e658-4b3c-93df-9d21dc1e19a6 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:16.257129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bd66edb7-e658-4b3c-93df-9d21dc1e19a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:16.293244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66edb7-e658-4b3c-93df-9d21dc1e19a6 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:16.332879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea138a-9cbb-4b99-8a8d-941b1470533c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361be69f1c0a41c18afd206bccf56447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:16.334396 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea138a-9cbb-4b99-8a8d-941b1470533c service placement] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:16.335603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea138a-9cbb-4b99-8a8d-941b1470533c service placement] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:16.335932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea138a-9cbb-4b99-8a8d-941b1470533c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:16.373761 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea138a-9cbb-4b99-8a8d-941b1470533c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:16.381709 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea138a-9cbb-4b99-8a8d-941b1470533c service placement] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:16.432622 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 861/1724] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:18.432239 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a2a3a-d83c-4131-bed6-4182546d15b9 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:18.434079 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-170a2a3a-d83c-4131-bed6-4182546d15b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:18.472762 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a2a3a-d83c-4131-bed6-4182546d15b9 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:18.500674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367cbd4-5b84-4fa5-8997-15d6821e5d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:18.501594 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367cbd4-5b84-4fa5-8997-15d6821e5d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:18.501916 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367cbd4-5b84-4fa5-8997-15d6821e5d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:18.502217 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367cbd4-5b84-4fa5-8997-15d6821e5d0e admin admin] PATH_INFO: `/v3/users/a4a21940483a4c009a5e19301cf6cafb` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:18.504405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367cbd4-5b84-4fa5-8997-15d6821e5d0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4a21940483a4c009a5e19301cf6cafb)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:18.510105 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367cbd4-5b84-4fa5-8997-15d6821e5d0e admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:18.621556 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea138a-9cbb-4b99-8a8d-941b1470533c service placement] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:18.623689 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9fea138a-9cbb-4b99-8a8d-941b1470533c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:18.676984 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea138a-9cbb-4b99-8a8d-941b1470533c service placement] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:18.718346 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b62a4-9cec-4546-964d-d7c32b110dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:18.721217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b62a4-9cec-4546-964d-d7c32b110dbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:18.724167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b62a4-9cec-4546-964d-d7c32b110dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:18.724564 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b62a4-9cec-4546-964d-d7c32b110dbd admin admin] PATH_INFO: `/v3/projects/3428f9ec1f0444e189b0d2c465f59419` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:18.728120 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b62a4-9cec-4546-964d-d7c32b110dbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3428f9ec1f0444e189b0d2c465f59419)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:18.735340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6367cbd4-5b84-4fa5-8997-15d6821e5d0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4a21940483a4c009a5e19301cf6cafb'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:18.736207 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b62a4-9cec-4546-964d-d7c32b110dbd admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:18.758632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6367cbd4-5b84-4fa5-8997-15d6821e5d0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4a21940483a4c009a5e19301cf6cafb'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:18.779007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6367cbd4-5b84-4fa5-8997-15d6821e5d0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4a21940483a4c009a5e19301cf6cafb'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:18.780120 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6367cbd4-5b84-4fa5-8997-15d6821e5d0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4a21940483a4c009a5e19301cf6cafb'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:18.824739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6367cbd4-5b84-4fa5-8997-15d6821e5d0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4a21940483a4c009a5e19301cf6cafb'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:18.862836 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 864/1725] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:18 2022] DELETE /identity/v3/users/a4a21940483a4c009a5e19301cf6cafb => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:18.892857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-678b62a4-9cec-4546-964d-d7c32b110dbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3428f9ec1f0444e189b0d2c465f59419', 'request_id': 'req-678b62a4-9cec-4546-964d-d7c32b110dbd'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:18.923606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-678b62a4-9cec-4546-964d-d7c32b110dbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3428f9ec1f0444e189b0d2c465f59419', 'request_id': 'req-678b62a4-9cec-4546-964d-d7c32b110dbd'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:18.938708 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-678b62a4-9cec-4546-964d-d7c32b110dbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3428f9ec1f0444e189b0d2c465f59419', 'request_id': 'req-678b62a4-9cec-4546-964d-d7c32b110dbd'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:18.965121 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-678b62a4-9cec-4546-964d-d7c32b110dbd admin admin] The token cache is being invalidate because project 3428f9ec1f0444e189b0d2c465f59419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:18.965972 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-678b62a4-9cec-4546-964d-d7c32b110dbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:18.967216 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-678b62a4-9cec-4546-964d-d7c32b110dbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:18.970124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 862/1726] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:18 2022] DELETE /identity/v3/projects/3428f9ec1f0444e189b0d2c465f59419 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:18.975225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ea4e6-5fce-4e47-9bd0-4f80288b3d50 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:18.975544 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ea4e6-5fce-4e47-9bd0-4f80288b3d50 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:18.975853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ea4e6-5fce-4e47-9bd0-4f80288b3d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:19.013310 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-6e7ea4e6-5fce-4e47-9bd0-4f80288b3d50 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:19.015150 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-6e7ea4e6-5fce-4e47-9bd0-4f80288b3d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:19.167265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 865/1727] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:19.174477 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc431c0f-b0c3-4656-be43-5840d72d9b4b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:19.174772 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc431c0f-b0c3-4656-be43-5840d72d9b4b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:19.175069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc431c0f-b0c3-4656-be43-5840d72d9b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:19.226168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-cc431c0f-b0c3-4656-be43-5840d72d9b4b None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:19.227357 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-cc431c0f-b0c3-4656-be43-5840d72d9b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:19.253511 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 863/1728] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:19.258270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ea4e6-5fce-4e47-9bd0-4f80288b3d50 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:19.279324 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ea4e6-5fce-4e47-9bd0-4f80288b3d50 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:19.299440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92c75b-840e-41d3-a13d-02d56c024333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:19.300151 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92c75b-840e-41d3-a13d-02d56c024333 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:19.300460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92c75b-840e-41d3-a13d-02d56c024333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:19.300735 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92c75b-840e-41d3-a13d-02d56c024333 admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:19.301504 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92c75b-840e-41d3-a13d-02d56c024333 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:38:19.301811 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92c75b-840e-41d3-a13d-02d56c024333 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:19.307372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92c75b-840e-41d3-a13d-02d56c024333 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:19.343215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 866/1729] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:38:19.345836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc431c0f-b0c3-4656-be43-5840d72d9b4b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:19.361949 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc431c0f-b0c3-4656-be43-5840d72d9b4b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:19.397122 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c01fc-fb04-46a4-a386-67147878c2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:19.397942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c01fc-fb04-46a4-a386-67147878c2ad admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:19.398123 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c01fc-fb04-46a4-a386-67147878c2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:19.398438 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c01fc-fb04-46a4-a386-67147878c2ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:19.399131 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c01fc-fb04-46a4-a386-67147878c2ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:19.401855 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c01fc-fb04-46a4-a386-67147878c2ad admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:19.469909 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 864/1730] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:19 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:19.480933 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92c75b-840e-41d3-a13d-02d56c024333 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:19.501650 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92c75b-840e-41d3-a13d-02d56c024333 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:19.577722 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:19.763950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c01fc-fb04-46a4-a386-67147878c2ad admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:19.766625 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d73c01fc-fb04-46a4-a386-67147878c2ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:19.768131 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14bb1e1-d254-473f-9d39-bbc011b5d324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:19.772495 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14bb1e1-d254-473f-9d39-bbc011b5d324 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:19.777051 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14bb1e1-d254-473f-9d39-bbc011b5d324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:19.777508 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14bb1e1-d254-473f-9d39-bbc011b5d324 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:19.778412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14bb1e1-d254-473f-9d39-bbc011b5d324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:19.794145 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14bb1e1-d254-473f-9d39-bbc011b5d324 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:19.810181 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c01fc-fb04-46a4-a386-67147878c2ad admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:19.844211 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa53702-00a4-43d4-8180-7f91bee335ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:19.845274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa53702-00a4-43d4-8180-7f91bee335ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:19.845657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa53702-00a4-43d4-8180-7f91bee335ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:19.852672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa53702-00a4-43d4-8180-7f91bee335ca admin admin] PATH_INFO: `/v3/projects/2b39d39669d94efcac1d3504acdfd22c` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:19.865258 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa53702-00a4-43d4-8180-7f91bee335ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b39d39669d94efcac1d3504acdfd22c)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:19.869077 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa53702-00a4-43d4-8180-7f91bee335ca admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:19.894581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 867/1731] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:19 2022] POST /identity/v3/users => generated 288 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:19.895497 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14bb1e1-d254-473f-9d39-bbc011b5d324 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:19.904560 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-c14bb1e1-d254-473f-9d39-bbc011b5d324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:19.943252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14bb1e1-d254-473f-9d39-bbc011b5d324 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:19.961324 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0aa53702-00a4-43d4-8180-7f91bee335ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b39d39669d94efcac1d3504acdfd22c', 'request_id': 'req-0aa53702-00a4-43d4-8180-7f91bee335ca'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:19.981914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0aa53702-00a4-43d4-8180-7f91bee335ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b39d39669d94efcac1d3504acdfd22c', 'request_id': 'req-0aa53702-00a4-43d4-8180-7f91bee335ca'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:19.993810 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0aa53702-00a4-43d4-8180-7f91bee335ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b39d39669d94efcac1d3504acdfd22c', 'request_id': 'req-0aa53702-00a4-43d4-8180-7f91bee335ca'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:19.997718 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:20.015675 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0aa53702-00a4-43d4-8180-7f91bee335ca admin admin] The token cache is being invalidate because project 2b39d39669d94efcac1d3504acdfd22c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:20.016623 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0aa53702-00a4-43d4-8180-7f91bee335ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:20.017528 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0aa53702-00a4-43d4-8180-7f91bee335ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:20.021197 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 865/1732] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:19 2022] DELETE /identity/v3/projects/2b39d39669d94efcac1d3504acdfd22c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:38:20.022374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa53702-00a4-43d4-8180-7f91bee335ca admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:20.040069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0aa53702-00a4-43d4-8180-7f91bee335ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:20.199382 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa53702-00a4-43d4-8180-7f91bee335ca admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:20.215092 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d1930-2497-4286-985c-d72f848f4d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:20.215824 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d1930-2497-4286-985c-d72f848f4d3b admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:20.216038 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d1930-2497-4286-985c-d72f848f4d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:20.216360 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d1930-2497-4286-985c-d72f848f4d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:20.217293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d1930-2497-4286-985c-d72f848f4d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:20.220226 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d1930-2497-4286-985c-d72f848f4d3b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:20.220904 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015754f-fb7b-43ab-9495-59066e342e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:20.223810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015754f-fb7b-43ab-9495-59066e342e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:20.224919 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015754f-fb7b-43ab-9495-59066e342e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:20.226294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015754f-fb7b-43ab-9495-59066e342e1e admin admin] PATH_INFO: `/v3/projects/90787cbf98cc4029858f9baf365eb4d5` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:20.259186 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 866/1733] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:20.259186 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d1930-2497-4286-985c-d72f848f4d3b admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:20.260774 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-080d1930-2497-4286-985c-d72f848f4d3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:20.277859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015754f-fb7b-43ab-9495-59066e342e1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=90787cbf98cc4029858f9baf365eb4d5)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:20.289522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015754f-fb7b-43ab-9495-59066e342e1e admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:20.313054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d1930-2497-4286-985c-d72f848f4d3b admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:20.350361 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf796b-5150-4dfd-a9c4-7f4c8cdcf1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:20.351268 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf796b-5150-4dfd-a9c4-7f4c8cdcf1fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:20.352050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf796b-5150-4dfd-a9c4-7f4c8cdcf1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:20.352330 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf796b-5150-4dfd-a9c4-7f4c8cdcf1fd admin admin] PATH_INFO: `/v3/users/70f1bdde9294473d83233ace443ce799` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:20.353991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf796b-5150-4dfd-a9c4-7f4c8cdcf1fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=70f1bdde9294473d83233ace443ce799)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:20.364670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf796b-5150-4dfd-a9c4-7f4c8cdcf1fd admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:20.446545 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9015754f-fb7b-43ab-9495-59066e342e1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90787cbf98cc4029858f9baf365eb4d5', 'request_id': 'req-9015754f-fb7b-43ab-9495-59066e342e1e'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:20.471373 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9015754f-fb7b-43ab-9495-59066e342e1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90787cbf98cc4029858f9baf365eb4d5', 'request_id': 'req-9015754f-fb7b-43ab-9495-59066e342e1e'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:20.492826 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9015754f-fb7b-43ab-9495-59066e342e1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90787cbf98cc4029858f9baf365eb4d5', 'request_id': 'req-9015754f-fb7b-43ab-9495-59066e342e1e'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:20.504812 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7baf796b-5150-4dfd-a9c4-7f4c8cdcf1fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70f1bdde9294473d83233ace443ce799'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:20.524091 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9015754f-fb7b-43ab-9495-59066e342e1e admin admin] The token cache is being invalidate because project 90787cbf98cc4029858f9baf365eb4d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:20.524668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9015754f-fb7b-43ab-9495-59066e342e1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:20.526251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9015754f-fb7b-43ab-9495-59066e342e1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:20.529789 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7baf796b-5150-4dfd-a9c4-7f4c8cdcf1fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70f1bdde9294473d83233ace443ce799'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:20.531550 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 868/1734] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:19 2022] DELETE /identity/v3/projects/90787cbf98cc4029858f9baf365eb4d5 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:20.532199 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7baf796b-5150-4dfd-a9c4-7f4c8cdcf1fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70f1bdde9294473d83233ace443ce799'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:20.533119 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015754f-fb7b-43ab-9495-59066e342e1e admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:20.533240 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7baf796b-5150-4dfd-a9c4-7f4c8cdcf1fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70f1bdde9294473d83233ace443ce799'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:20.535529 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-9015754f-fb7b-43ab-9495-59066e342e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:20.582779 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7baf796b-5150-4dfd-a9c4-7f4c8cdcf1fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70f1bdde9294473d83233ace443ce799'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:20.610107 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 867/1735] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:20 2022] DELETE /identity/v3/users/70f1bdde9294473d83233ace443ce799 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:20.611223 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf796b-5150-4dfd-a9c4-7f4c8cdcf1fd admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:20.614208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-7baf796b-5150-4dfd-a9c4-7f4c8cdcf1fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:20.635468 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:20.715981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015754f-fb7b-43ab-9495-59066e342e1e admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:20.849041 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf796b-5150-4dfd-a9c4-7f4c8cdcf1fd admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:20.859804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f57fce-37b9-49d6-96d7-fb61ccba4b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:20.861013 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f57fce-37b9-49d6-96d7-fb61ccba4b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:20.861447 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f57fce-37b9-49d6-96d7-fb61ccba4b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:20.862053 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f57fce-37b9-49d6-96d7-fb61ccba4b4f admin admin] PATH_INFO: `/v3/users/3b978f134fe04792978a8790365329d1` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:20.863708 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f57fce-37b9-49d6-96d7-fb61ccba4b4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b978f134fe04792978a8790365329d1)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:20.865934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f57fce-37b9-49d6-96d7-fb61ccba4b4f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:20.872806 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:20.911704 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d4a26-40bb-4893-81a8-bc3d8e80b220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:20.914602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d4a26-40bb-4893-81a8-bc3d8e80b220 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:20.914974 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d4a26-40bb-4893-81a8-bc3d8e80b220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:20.915381 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d4a26-40bb-4893-81a8-bc3d8e80b220 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:20.915968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d4a26-40bb-4893-81a8-bc3d8e80b220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:20.926134 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d4a26-40bb-4893-81a8-bc3d8e80b220 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:20.974499 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 869/1736] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:20.977251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d4a26-40bb-4893-81a8-bc3d8e80b220 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:20.981864 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-0a1d4a26-40bb-4893-81a8-bc3d8e80b220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:21.010810 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c8f57fce-37b9-49d6-96d7-fb61ccba4b4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b978f134fe04792978a8790365329d1'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:21.029583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d4a26-40bb-4893-81a8-bc3d8e80b220 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:21.037065 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c8f57fce-37b9-49d6-96d7-fb61ccba4b4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b978f134fe04792978a8790365329d1'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:21.040267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c8f57fce-37b9-49d6-96d7-fb61ccba4b4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b978f134fe04792978a8790365329d1'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:21.043315 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c8f57fce-37b9-49d6-96d7-fb61ccba4b4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b978f134fe04792978a8790365329d1'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:21.044281 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71861a7a-60ca-41f4-a240-acad577a8dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:21.047342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71861a7a-60ca-41f4-a240-acad577a8dca admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:21.048157 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71861a7a-60ca-41f4-a240-acad577a8dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:21.049301 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71861a7a-60ca-41f4-a240-acad577a8dca admin admin] PATH_INFO: `/v3/projects/4d89f821ea534751b38d212cf85490fa/users/e84e9b531eca460682dc3b5b730b9a14/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:21.080066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c8f57fce-37b9-49d6-96d7-fb61ccba4b4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b978f134fe04792978a8790365329d1'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:21.120527 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71861a7a-60ca-41f4-a240-acad577a8dca admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d89f821ea534751b38d212cf85490fa, user_id=e84e9b531eca460682dc3b5b730b9a14, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:21.120985 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 868/1737] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:20 2022] DELETE /identity/v3/users/3b978f134fe04792978a8790365329d1 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:21.130707 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71861a7a-60ca-41f4-a240-acad577a8dca admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:21.167640 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 870/1738] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:20 2022] PUT /identity/v3/projects/4d89f821ea534751b38d212cf85490fa/users/e84e9b531eca460682dc3b5b730b9a14/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:21.178518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc892d9-845d-4607-a6c9-e828be1b0d32 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:21.178950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc892d9-845d-4607-a6c9-e828be1b0d32 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:21.179237 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc892d9-845d-4607-a6c9-e828be1b0d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:21.216963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d18dd8-df75-4047-ac55-758ab82496e1 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:21.216963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d18dd8-df75-4047-ac55-758ab82496e1 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:21.217240 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d18dd8-df75-4047-ac55-758ab82496e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:21.268117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-c7d18dd8-df75-4047-ac55-758ab82496e1 None None] MFA Rules not processed for user `f269e146c844480c822347c3a38d103b`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:21.269912 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-c7d18dd8-df75-4047-ac55-758ab82496e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:21.286401 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-dcc892d9-845d-4607-a6c9-e828be1b0d32 None None] MFA Rules not processed for user `e84e9b531eca460682dc3b5b730b9a14`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:21.288023 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-dcc892d9-845d-4607-a6c9-e828be1b0d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:21.520673 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 869/1739] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:21.534168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dba06-9b49-4eed-ab1a-d20c09789e6b None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:21.535193 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dba06-9b49-4eed-ab1a-d20c09789e6b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:21.536019 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dba06-9b49-4eed-ab1a-d20c09789e6b None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:21.542634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 870/1740] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jun 6 10:38:21 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:38:21.546908 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 871/1741] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:21.554130 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dba06-9b49-4eed-ab1a-d20c09789e6b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:21.557777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f507505-9ce9-4ad9-9af0-62543dfcce90 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:21.558726 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f507505-9ce9-4ad9-9af0-62543dfcce90 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:21.560964 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b21dba06-9b49-4eed-ab1a-d20c09789e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:21.568277 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f507505-9ce9-4ad9-9af0-62543dfcce90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:21.622676 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-4f507505-9ce9-4ad9-9af0-62543dfcce90 None None] MFA Rules not processed for user `f269e146c844480c822347c3a38d103b`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:21.624425 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-4f507505-9ce9-4ad9-9af0-62543dfcce90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:21.661940 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 872/1742] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:21.668948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f507505-9ce9-4ad9-9af0-62543dfcce90 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:21.670072 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-4f507505-9ce9-4ad9-9af0-62543dfcce90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:21.809863 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:21.816966 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dba06-9b49-4eed-ab1a-d20c09789e6b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:21.846374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fd96a-4641-4c88-87c0-482d40b9d03d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:21.847937 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fd96a-4641-4c88-87c0-482d40b9d03d service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:21.849946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fd96a-4641-4c88-87c0-482d40b9d03d service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:21.851253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fd96a-4641-4c88-87c0-482d40b9d03d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:21.879126 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fd96a-4641-4c88-87c0-482d40b9d03d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:21.887728 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fd96a-4641-4c88-87c0-482d40b9d03d service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:21.895533 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f507505-9ce9-4ad9-9af0-62543dfcce90 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:21.899707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:21.935963 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 871/1743] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:21.938041 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318b0ad-4c3f-4e58-b816-cbdbcc31e721 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:21.938333 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318b0ad-4c3f-4e58-b816-cbdbcc31e721 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:21.938617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318b0ad-4c3f-4e58-b816-cbdbcc31e721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:21.940592 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a9719-e4f5-4d60-98c2-7df2354c69fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:21.941305 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a9719-e4f5-4d60-98c2-7df2354c69fb service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:21.942366 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a9719-e4f5-4d60-98c2-7df2354c69fb service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:21.942366 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a9719-e4f5-4d60-98c2-7df2354c69fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:21.980351 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a9719-e4f5-4d60-98c2-7df2354c69fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:21.987667 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a9719-e4f5-4d60-98c2-7df2354c69fb service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:22.033547 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-8318b0ad-4c3f-4e58-b816-cbdbcc31e721 None None] MFA Rules not processed for user `ed8a40e4f1034a8e99d0eb505e448db9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:22.035866 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 873/1744] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:38:22.039900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-8318b0ad-4c3f-4e58-b816-cbdbcc31e721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:22.041803 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a9719-e4f5-4d60-98c2-7df2354c69fb service nova] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:22.045368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-180a9719-e4f5-4d60-98c2-7df2354c69fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:22.081583 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 872/1745] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jun 6 10:38:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:22.248727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318b0ad-4c3f-4e58-b816-cbdbcc31e721 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:22.276333 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318b0ad-4c3f-4e58-b816-cbdbcc31e721 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:22.312079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6cef7-17d9-44e4-8c03-b477190799bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:22.319503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6cef7-17d9-44e4-8c03-b477190799bc service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:22.320164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6cef7-17d9-44e4-8c03-b477190799bc service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:22.320565 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6cef7-17d9-44e4-8c03-b477190799bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:22.340720 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6cef7-17d9-44e4-8c03-b477190799bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:22.345833 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6cef7-17d9-44e4-8c03-b477190799bc service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:22.404317 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 873/1746] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:22.407239 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a9719-e4f5-4d60-98c2-7df2354c69fb service nova] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:22.433604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515b9ea-156f-40bb-9e12-544a0120d3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:22.434495 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515b9ea-156f-40bb-9e12-544a0120d3fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:22.434785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515b9ea-156f-40bb-9e12-544a0120d3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:22.435042 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515b9ea-156f-40bb-9e12-544a0120d3fb admin admin] PATH_INFO: `/v3/projects/5c7cf5d7d30245dfbf08c63be3a28dc7` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:22.437893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515b9ea-156f-40bb-9e12-544a0120d3fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c7cf5d7d30245dfbf08c63be3a28dc7)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:22.439977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515b9ea-156f-40bb-9e12-544a0120d3fb admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:22.609852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6515b9ea-156f-40bb-9e12-544a0120d3fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c7cf5d7d30245dfbf08c63be3a28dc7', 'request_id': 'req-6515b9ea-156f-40bb-9e12-544a0120d3fb'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:22.624736 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6515b9ea-156f-40bb-9e12-544a0120d3fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c7cf5d7d30245dfbf08c63be3a28dc7', 'request_id': 'req-6515b9ea-156f-40bb-9e12-544a0120d3fb'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:22.642365 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6515b9ea-156f-40bb-9e12-544a0120d3fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c7cf5d7d30245dfbf08c63be3a28dc7', 'request_id': 'req-6515b9ea-156f-40bb-9e12-544a0120d3fb'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:22.675920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6515b9ea-156f-40bb-9e12-544a0120d3fb admin admin] The token cache is being invalidate because project 5c7cf5d7d30245dfbf08c63be3a28dc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:22.676218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6515b9ea-156f-40bb-9e12-544a0120d3fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:22.677516 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6515b9ea-156f-40bb-9e12-544a0120d3fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:22.679224 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 874/1747] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:22 2022] DELETE /identity/v3/projects/5c7cf5d7d30245dfbf08c63be3a28dc7 => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:22.683659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6cef7-17d9-44e4-8c03-b477190799bc service nova] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:22.712372 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6cef7-17d9-44e4-8c03-b477190799bc service nova] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:22.893661 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943e4d3-ea36-49e1-9a51-2f707a549295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:22.897002 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943e4d3-ea36-49e1-9a51-2f707a549295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:22.897517 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943e4d3-ea36-49e1-9a51-2f707a549295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:22.898002 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943e4d3-ea36-49e1-9a51-2f707a549295 admin admin] PATH_INFO: `/v3/users/465acb54186f4627a44d8fa7379decd6` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:22.899504 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943e4d3-ea36-49e1-9a51-2f707a549295 admin admin] RBAC: Authorizing `identity:delete_user(user_id=465acb54186f4627a44d8fa7379decd6)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:22.901783 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943e4d3-ea36-49e1-9a51-2f707a549295 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:23.070380 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2943e4d3-ea36-49e1-9a51-2f707a549295 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '465acb54186f4627a44d8fa7379decd6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:23.094797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2943e4d3-ea36-49e1-9a51-2f707a549295 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '465acb54186f4627a44d8fa7379decd6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:23.101225 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2943e4d3-ea36-49e1-9a51-2f707a549295 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '465acb54186f4627a44d8fa7379decd6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:23.102273 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2943e4d3-ea36-49e1-9a51-2f707a549295 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '465acb54186f4627a44d8fa7379decd6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:23.132371 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2943e4d3-ea36-49e1-9a51-2f707a549295 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '465acb54186f4627a44d8fa7379decd6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:23.205632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 874/1748] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:22 2022] DELETE /identity/v3/users/465acb54186f4627a44d8fa7379decd6 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:23.880668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515b9ea-156f-40bb-9e12-544a0120d3fb admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:23.882129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-6515b9ea-156f-40bb-9e12-544a0120d3fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:23.913387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515b9ea-156f-40bb-9e12-544a0120d3fb admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:24.192909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:24.220928 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa051e5-51c8-4d1b-80b8-6643adda3a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:24.231654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa051e5-51c8-4d1b-80b8-6643adda3a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:24.234735 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa051e5-51c8-4d1b-80b8-6643adda3a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:24.235137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa051e5-51c8-4d1b-80b8-6643adda3a3f admin admin] PATH_INFO: `/v3/projects/73f77838dcb2424081c1f0d20647a0ac` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:24.243821 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa051e5-51c8-4d1b-80b8-6643adda3a3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=73f77838dcb2424081c1f0d20647a0ac)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:24.246378 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa051e5-51c8-4d1b-80b8-6643adda3a3f admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:24.366328 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943e4d3-ea36-49e1-9a51-2f707a549295 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:24.369289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2943e4d3-ea36-49e1-9a51-2f707a549295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:24.424903 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7aa051e5-51c8-4d1b-80b8-6643adda3a3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73f77838dcb2424081c1f0d20647a0ac', 'request_id': 'req-7aa051e5-51c8-4d1b-80b8-6643adda3a3f'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:24.427202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943e4d3-ea36-49e1-9a51-2f707a549295 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:24.452475 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7aa051e5-51c8-4d1b-80b8-6643adda3a3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73f77838dcb2424081c1f0d20647a0ac', 'request_id': 'req-7aa051e5-51c8-4d1b-80b8-6643adda3a3f'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:24.475908 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7aa051e5-51c8-4d1b-80b8-6643adda3a3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73f77838dcb2424081c1f0d20647a0ac', 'request_id': 'req-7aa051e5-51c8-4d1b-80b8-6643adda3a3f'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:24.520973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7aa051e5-51c8-4d1b-80b8-6643adda3a3f admin admin] The token cache is being invalidate because project 73f77838dcb2424081c1f0d20647a0ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:24.521439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7aa051e5-51c8-4d1b-80b8-6643adda3a3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:24.524960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7aa051e5-51c8-4d1b-80b8-6643adda3a3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:24.529285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 875/1749] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:23 2022] DELETE /identity/v3/projects/73f77838dcb2424081c1f0d20647a0ac => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:24.536559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722a5ed-f432-40f1-ae5b-3ea22645888b None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:24.537038 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722a5ed-f432-40f1-ae5b-3ea22645888b None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:24.537512 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722a5ed-f432-40f1-ae5b-3ea22645888b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:24.538620 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:24.548570 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19094cd4-5d76-4b4e-b813-4f2154f911ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:24.550676 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19094cd4-5d76-4b4e-b813-4f2154f911ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:24.551179 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19094cd4-5d76-4b4e-b813-4f2154f911ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:24.551603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19094cd4-5d76-4b4e-b813-4f2154f911ae admin admin] PATH_INFO: `/v3/users/e84e9b531eca460682dc3b5b730b9a14` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:24.553937 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19094cd4-5d76-4b4e-b813-4f2154f911ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=e84e9b531eca460682dc3b5b730b9a14)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:24.556519 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19094cd4-5d76-4b4e-b813-4f2154f911ae admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:24.619102 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-c722a5ed-f432-40f1-ae5b-3ea22645888b None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:24.632628 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-c722a5ed-f432-40f1-ae5b-3ea22645888b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:24.646405 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-19094cd4-5d76-4b4e-b813-4f2154f911ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e84e9b531eca460682dc3b5b730b9a14'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:24.684435 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-19094cd4-5d76-4b4e-b813-4f2154f911ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e84e9b531eca460682dc3b5b730b9a14'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:24.685373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-19094cd4-5d76-4b4e-b813-4f2154f911ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e84e9b531eca460682dc3b5b730b9a14'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:24.686802 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-19094cd4-5d76-4b4e-b813-4f2154f911ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e84e9b531eca460682dc3b5b730b9a14'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:24.706984 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 876/1750] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:38:24.709066 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e611e-9c93-4f41-a508-b1bd1638b6cd None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:24.709376 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e611e-9c93-4f41-a508-b1bd1638b6cd None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:24.709626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e611e-9c93-4f41-a508-b1bd1638b6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:24.749324 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-19094cd4-5d76-4b4e-b813-4f2154f911ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e84e9b531eca460682dc3b5b730b9a14'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:24.774057 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-397e611e-9c93-4f41-a508-b1bd1638b6cd None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:24.777532 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-397e611e-9c93-4f41-a508-b1bd1638b6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:24.784337 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 875/1751] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:24 2022] DELETE /identity/v3/users/e84e9b531eca460682dc3b5b730b9a14 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:24.788672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0661f8-30c8-45cd-a4cb-c2f536f56de5 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:24.788955 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0661f8-30c8-45cd-a4cb-c2f536f56de5 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:24.789225 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0661f8-30c8-45cd-a4cb-c2f536f56de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:24.802502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 877/1752] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:24.804918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e611e-9c93-4f41-a508-b1bd1638b6cd None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:24.852557 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e611e-9c93-4f41-a508-b1bd1638b6cd None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:24.854644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-6b0661f8-30c8-45cd-a4cb-c2f536f56de5 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:24.860659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6b0661f8-30c8-45cd-a4cb-c2f536f56de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:24.966404 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 876/1753] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:25.017766 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88b0ab-b143-42bb-8bd5-83f88313fa54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:25.020064 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88b0ab-b143-42bb-8bd5-83f88313fa54 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:25.020644 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88b0ab-b143-42bb-8bd5-83f88313fa54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:25.021832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88b0ab-b143-42bb-8bd5-83f88313fa54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:25.022818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88b0ab-b143-42bb-8bd5-83f88313fa54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:38:25.024564 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88b0ab-b143-42bb-8bd5-83f88313fa54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:25.026456 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88b0ab-b143-42bb-8bd5-83f88313fa54 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:25.035242 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0661f8-30c8-45cd-a4cb-c2f536f56de5 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:25.037534 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6b0661f8-30c8-45cd-a4cb-c2f536f56de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:25.101074 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 878/1754] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:38:25.106504 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88b0ab-b143-42bb-8bd5-83f88313fa54 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:25.131586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88b0ab-b143-42bb-8bd5-83f88313fa54 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:25.145980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c5b146-4578-4258-95be-f5dd400ea680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:25.146892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5b146-4578-4258-95be-f5dd400ea680 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:25.147179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5b146-4578-4258-95be-f5dd400ea680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:25.147527 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5b146-4578-4258-95be-f5dd400ea680 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:25.148446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c5b146-4578-4258-95be-f5dd400ea680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:25.151181 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c5b146-4578-4258-95be-f5dd400ea680 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:25.313853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 879/1755] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:25 2022] POST /identity/v3/projects => generated 371 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:25.316340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c5b146-4578-4258-95be-f5dd400ea680 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:25.397616 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0661f8-30c8-45cd-a4cb-c2f536f56de5 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:25.488865 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:25.533447 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:25.557706 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93da5d-ab4b-465b-8058-fe30d25a214e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:25.558390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93da5d-ab4b-465b-8058-fe30d25a214e service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:25.558701 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93da5d-ab4b-465b-8058-fe30d25a214e service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:25.558991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93da5d-ab4b-465b-8058-fe30d25a214e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:25.568708 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c5b146-4578-4258-95be-f5dd400ea680 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:25.584634 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0812d-eff3-48e8-bc4d-27a6c037ceb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:25.585365 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0812d-eff3-48e8-bc4d-27a6c037ceb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:25.585655 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0812d-eff3-48e8-bc4d-27a6c037ceb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:25.586000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0812d-eff3-48e8-bc4d-27a6c037ceb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:25.593284 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93da5d-ab4b-465b-8058-fe30d25a214e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:25.593797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0812d-eff3-48e8-bc4d-27a6c037ceb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:25.597551 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93da5d-ab4b-465b-8058-fe30d25a214e service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:25.600082 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0812d-eff3-48e8-bc4d-27a6c037ceb5 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:25.618938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 877/1756] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:25.645269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 880/1757] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:25 2022] POST /identity/v3/users => generated 286 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:25.651415 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93da5d-ab4b-465b-8058-fe30d25a214e service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:25.669332 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93da5d-ab4b-465b-8058-fe30d25a214e service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:25.673827 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72a4d3-39cd-41ad-9825-4dc2bb3d542a None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:25.674122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72a4d3-39cd-41ad-9825-4dc2bb3d542a None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:25.674406 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72a4d3-39cd-41ad-9825-4dc2bb3d542a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:25.700043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c01dd4b-dd0d-4eb4-8ab3-cd9be6be8ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:25.701381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c01dd4b-dd0d-4eb4-8ab3-cd9be6be8ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:25.701698 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c01dd4b-dd0d-4eb4-8ab3-cd9be6be8ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:25.706262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c01dd4b-dd0d-4eb4-8ab3-cd9be6be8ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:25.707255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c01dd4b-dd0d-4eb4-8ab3-cd9be6be8ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:25.712151 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c01dd4b-dd0d-4eb4-8ab3-cd9be6be8ba3 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:25.789862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 878/1758] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:25.791095 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c01dd4b-dd0d-4eb4-8ab3-cd9be6be8ba3 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:25.810916 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-bc72a4d3-39cd-41ad-9825-4dc2bb3d542a None None] MFA Rules not processed for user `ea1482ba53554f71865cea46ffbac23c`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:25.816971 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-bc72a4d3-39cd-41ad-9825-4dc2bb3d542a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:25.837107 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c01dd4b-dd0d-4eb4-8ab3-cd9be6be8ba3 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:25.856712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99e95e-142d-4892-96fd-586f6ab173f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:25.859225 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99e95e-142d-4892-96fd-586f6ab173f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:25.862617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99e95e-142d-4892-96fd-586f6ab173f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:25.863103 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99e95e-142d-4892-96fd-586f6ab173f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:25.864246 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99e95e-142d-4892-96fd-586f6ab173f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:25.873080 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99e95e-142d-4892-96fd-586f6ab173f0 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:25.924044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 879/1759] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:25.924931 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99e95e-142d-4892-96fd-586f6ab173f0 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:25.935418 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fe99e95e-142d-4892-96fd-586f6ab173f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:25.990620 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99e95e-142d-4892-96fd-586f6ab173f0 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:26.010368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbb0a4-6369-4aa5-8351-1ba70e22cb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:26.013013 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbb0a4-6369-4aa5-8351-1ba70e22cb43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:26.013651 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbb0a4-6369-4aa5-8351-1ba70e22cb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:26.013944 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbb0a4-6369-4aa5-8351-1ba70e22cb43 admin admin] PATH_INFO: `/v3/projects/03391f861ab5454b95e3f149c65651f7` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:26.026161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acbb0a4-6369-4aa5-8351-1ba70e22cb43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03391f861ab5454b95e3f149c65651f7)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:26.032588 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acbb0a4-6369-4aa5-8351-1ba70e22cb43 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:26.111018 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 881/1760] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:38:26.117257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc72a4d3-39cd-41ad-9825-4dc2bb3d542a None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:26.186484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc72a4d3-39cd-41ad-9825-4dc2bb3d542a None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:26.210322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d7ba2-f132-4f02-a0a0-344981640233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:26.213995 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d7ba2-f132-4f02-a0a0-344981640233 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:26.215748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d7ba2-f132-4f02-a0a0-344981640233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:26.216232 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d7ba2-f132-4f02-a0a0-344981640233 admin admin] PATH_INFO: `/v3/projects/09aef592ea7042ecbc4687aafb0f2469/users/c47d8ba602c54febac5f6203f9517351/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:26.263970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4acbb0a4-6369-4aa5-8351-1ba70e22cb43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03391f861ab5454b95e3f149c65651f7', 'request_id': 'req-4acbb0a4-6369-4aa5-8351-1ba70e22cb43'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:26.293726 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4acbb0a4-6369-4aa5-8351-1ba70e22cb43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03391f861ab5454b95e3f149c65651f7', 'request_id': 'req-4acbb0a4-6369-4aa5-8351-1ba70e22cb43'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:26.296458 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d7ba2-f132-4f02-a0a0-344981640233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09aef592ea7042ecbc4687aafb0f2469, user_id=c47d8ba602c54febac5f6203f9517351, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:26.311770 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d7ba2-f132-4f02-a0a0-344981640233 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:26.328083 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4acbb0a4-6369-4aa5-8351-1ba70e22cb43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03391f861ab5454b95e3f149c65651f7', 'request_id': 'req-4acbb0a4-6369-4aa5-8351-1ba70e22cb43'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:26.381332 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4acbb0a4-6369-4aa5-8351-1ba70e22cb43 admin admin] The token cache is being invalidate because project 03391f861ab5454b95e3f149c65651f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:26.385879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4acbb0a4-6369-4aa5-8351-1ba70e22cb43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:26.388516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4acbb0a4-6369-4aa5-8351-1ba70e22cb43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:26.393220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 882/1761] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:26 2022] PUT /identity/v3/projects/09aef592ea7042ecbc4687aafb0f2469/users/c47d8ba602c54febac5f6203f9517351/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:26.398407 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d7ba2-f132-4f02-a0a0-344981640233 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:26.402489 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ca2d7ba2-f132-4f02-a0a0-344981640233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:26.403570 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 880/1762] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:25 2022] DELETE /identity/v3/projects/03391f861ab5454b95e3f149c65651f7 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:26.407957 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbb0a4-6369-4aa5-8351-1ba70e22cb43 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:26.409369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4acbb0a4-6369-4aa5-8351-1ba70e22cb43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:26.591283 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbb0a4-6369-4aa5-8351-1ba70e22cb43 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:26.673911 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:26.675293 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d7ba2-f132-4f02-a0a0-344981640233 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:26.692593 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c5aa3-f798-453e-a8c7-d66bd0e2f2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:26.696903 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c5aa3-f798-453e-a8c7-d66bd0e2f2de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:26.698249 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c5aa3-f798-453e-a8c7-d66bd0e2f2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:26.698726 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c5aa3-f798-453e-a8c7-d66bd0e2f2de admin admin] PATH_INFO: `/v3/projects/ed9758dd444c45db9d78dbd04810507c` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:26.706838 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c5aa3-f798-453e-a8c7-d66bd0e2f2de admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed9758dd444c45db9d78dbd04810507c)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:26.718951 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c5aa3-f798-453e-a8c7-d66bd0e2f2de admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:26.749075 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d979d-c2c9-4d63-8928-a92ad8e6d206 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:26.749826 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d979d-c2c9-4d63-8928-a92ad8e6d206 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:26.750230 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d979d-c2c9-4d63-8928-a92ad8e6d206 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:26.750511 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d979d-c2c9-4d63-8928-a92ad8e6d206 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:26.756896 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-007d979d-c2c9-4d63-8928-a92ad8e6d206 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:26.916554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:26.933435 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e83c5aa3-f798-453e-a8c7-d66bd0e2f2de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed9758dd444c45db9d78dbd04810507c', 'request_id': 'req-e83c5aa3-f798-453e-a8c7-d66bd0e2f2de'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:26.944084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d979d-c2c9-4d63-8928-a92ad8e6d206 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:26.946500 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d979d-c2c9-4d63-8928-a92ad8e6d206 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:26.972047 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e83c5aa3-f798-453e-a8c7-d66bd0e2f2de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed9758dd444c45db9d78dbd04810507c', 'request_id': 'req-e83c5aa3-f798-453e-a8c7-d66bd0e2f2de'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:26.998713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e83c5aa3-f798-453e-a8c7-d66bd0e2f2de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed9758dd444c45db9d78dbd04810507c', 'request_id': 'req-e83c5aa3-f798-453e-a8c7-d66bd0e2f2de'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:26.999631 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 881/1763] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:27.018630 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d45e4-dc27-4e74-a871-3452740d29c8 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:27.019020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d45e4-dc27-4e74-a871-3452740d29c8 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:27.019249 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d45e4-dc27-4e74-a871-3452740d29c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:27.040889 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e83c5aa3-f798-453e-a8c7-d66bd0e2f2de admin admin] The token cache is being invalidate because project ed9758dd444c45db9d78dbd04810507c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:27.042345 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e83c5aa3-f798-453e-a8c7-d66bd0e2f2de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:27.049109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e83c5aa3-f798-453e-a8c7-d66bd0e2f2de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:27.056098 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 883/1764] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:26 2022] DELETE /identity/v3/projects/ed9758dd444c45db9d78dbd04810507c => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:27.059285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05debf-6511-4ff6-ac2e-cd2184321cad None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:27.059674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05debf-6511-4ff6-ac2e-cd2184321cad None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:27.060238 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05debf-6511-4ff6-ac2e-cd2184321cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:27.089345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2e8d45e4-dc27-4e74-a871-3452740d29c8 None None] MFA Rules not processed for user `f269e146c844480c822347c3a38d103b`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:27.090687 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2e8d45e4-dc27-4e74-a871-3452740d29c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:27.141605 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-be05debf-6511-4ff6-ac2e-cd2184321cad None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:27.142897 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-be05debf-6511-4ff6-ac2e-cd2184321cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:27.342702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 884/1765] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:27.350353 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58179827-c4cc-4ef0-a242-3b7807e32f67 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:27.350667 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58179827-c4cc-4ef0-a242-3b7807e32f67 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:27.353447 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58179827-c4cc-4ef0-a242-3b7807e32f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:27.360351 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 882/1766] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:27.361749 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949355f-0b50-4882-a30f-15b82b9f7169 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:27.362004 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949355f-0b50-4882-a30f-15b82b9f7169 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:27.362340 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949355f-0b50-4882-a30f-15b82b9f7169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:27.421767 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-f949355f-0b50-4882-a30f-15b82b9f7169 None None] MFA Rules not processed for user `f6b7d82c999646b6b6b96d565f0e3f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:27.423036 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f949355f-0b50-4882-a30f-15b82b9f7169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:27.450460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-58179827-c4cc-4ef0-a242-3b7807e32f67 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:27.452765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-58179827-c4cc-4ef0-a242-3b7807e32f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:27.501379 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 885/1767] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:27.508906 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb447ff6-b057-4768-8cb5-5dd3792929b1 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:27.508906 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb447ff6-b057-4768-8cb5-5dd3792929b1 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:27.508906 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb447ff6-b057-4768-8cb5-5dd3792929b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:27.578474 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-fb447ff6-b057-4768-8cb5-5dd3792929b1 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:27.579813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-fb447ff6-b057-4768-8cb5-5dd3792929b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:27.615620 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 886/1768] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:27.617829 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f3586-2383-437d-86ef-1e9e7c7c481d None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:27.618163 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f3586-2383-437d-86ef-1e9e7c7c481d None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:27.618584 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f3586-2383-437d-86ef-1e9e7c7c481d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:27.648738 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 883/1769] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:27.650571 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28444503-8e4f-4568-8733-8ce4287b3f03 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:27.651152 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28444503-8e4f-4568-8733-8ce4287b3f03 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:27.651613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28444503-8e4f-4568-8733-8ce4287b3f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:27.697377 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-b76f3586-2383-437d-86ef-1e9e7c7c481d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:27.698748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-b76f3586-2383-437d-86ef-1e9e7c7c481d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:27.718700 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-28444503-8e4f-4568-8733-8ce4287b3f03 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:27.722540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-28444503-8e4f-4568-8733-8ce4287b3f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:27.725069 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 887/1770] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:27.729899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f3586-2383-437d-86ef-1e9e7c7c481d None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:27.753667 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 884/1771] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:27.757431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f3586-2383-437d-86ef-1e9e7c7c481d None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:27.757686 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28444503-8e4f-4568-8733-8ce4287b3f03 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:27.760544 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-28444503-8e4f-4568-8733-8ce4287b3f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:27.778327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c829f-91bb-4a68-b713-aa103c76fa18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:27.779207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c829f-91bb-4a68-b713-aa103c76fa18 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:27.779537 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c829f-91bb-4a68-b713-aa103c76fa18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:27.780467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c829f-91bb-4a68-b713-aa103c76fa18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:27.781436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c829f-91bb-4a68-b713-aa103c76fa18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:38:27.781721 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c829f-91bb-4a68-b713-aa103c76fa18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:27.783615 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c829f-91bb-4a68-b713-aa103c76fa18 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:27.820471 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:27.820565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 888/1772] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:38:27.821095 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c829f-91bb-4a68-b713-aa103c76fa18 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:27.848241 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c829f-91bb-4a68-b713-aa103c76fa18 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:27.859962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b26755-0013-45b3-9371-d89ed9e033f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:27.860684 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b26755-0013-45b3-9371-d89ed9e033f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:27.861632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b26755-0013-45b3-9371-d89ed9e033f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:27.861632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b26755-0013-45b3-9371-d89ed9e033f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:27.862149 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b26755-0013-45b3-9371-d89ed9e033f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:38:27.862497 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b26755-0013-45b3-9371-d89ed9e033f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:27.866898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b26755-0013-45b3-9371-d89ed9e033f5 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:27.894444 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 889/1773] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:38:27.895317 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b26755-0013-45b3-9371-d89ed9e033f5 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:27.921521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b26755-0013-45b3-9371-d89ed9e033f5 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:27.932885 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf76f6-b2fa-4b33-b908-470b86454b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:27.933586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf76f6-b2fa-4b33-b908-470b86454b6b admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:27.933910 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf76f6-b2fa-4b33-b908-470b86454b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:27.934283 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf76f6-b2fa-4b33-b908-470b86454b6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:27.934893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf76f6-b2fa-4b33-b908-470b86454b6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:27.942593 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf76f6-b2fa-4b33-b908-470b86454b6b admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:27.957182 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28444503-8e4f-4568-8733-8ce4287b3f03 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:27.965249 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda67990-7bc6-49be-9553-74d598a6f420 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:27.966243 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda67990-7bc6-49be-9553-74d598a6f420 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:27.966531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda67990-7bc6-49be-9553-74d598a6f420 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:27.966809 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda67990-7bc6-49be-9553-74d598a6f420 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:27.990902 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda67990-7bc6-49be-9553-74d598a6f420 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:28.007172 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda67990-7bc6-49be-9553-74d598a6f420 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:28.044193 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 885/1774] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:28.046694 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda67990-7bc6-49be-9553-74d598a6f420 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:28.058943 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda67990-7bc6-49be-9553-74d598a6f420 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:28.069947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 890/1775] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:27 2022] POST /identity/v3/projects => generated 371 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:28.074809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf76f6-b2fa-4b33-b908-470b86454b6b admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:28.152401 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76892288-11cd-4ed1-894b-154a31567e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:28.153078 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76892288-11cd-4ed1-894b-154a31567e4f admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:28.153366 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76892288-11cd-4ed1-894b-154a31567e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:28.153639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76892288-11cd-4ed1-894b-154a31567e4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:28.155601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76892288-11cd-4ed1-894b-154a31567e4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:28.157567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76892288-11cd-4ed1-894b-154a31567e4f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:28.194269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf76f6-b2fa-4b33-b908-470b86454b6b admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:28.194941 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 886/1776] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:28 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:28.198180 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76892288-11cd-4ed1-894b-154a31567e4f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:28.207954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:28.313666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76892288-11cd-4ed1-894b-154a31567e4f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:28.320070 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736ea31-b41c-4495-87a6-b4f09dfcc188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:28.320780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736ea31-b41c-4495-87a6-b4f09dfcc188 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:28.321310 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736ea31-b41c-4495-87a6-b4f09dfcc188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:28.321590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736ea31-b41c-4495-87a6-b4f09dfcc188 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:28.322288 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736ea31-b41c-4495-87a6-b4f09dfcc188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:28.324978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736ea31-b41c-4495-87a6-b4f09dfcc188 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:28.332721 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d42cec-38c1-4ce2-813a-b80d594c0798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:28.334127 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d42cec-38c1-4ce2-813a-b80d594c0798 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:28.334693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d42cec-38c1-4ce2-813a-b80d594c0798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:28.334810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d42cec-38c1-4ce2-813a-b80d594c0798 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:28.335544 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d42cec-38c1-4ce2-813a-b80d594c0798 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:28.338009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d42cec-38c1-4ce2-813a-b80d594c0798 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:28.358667 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 887/1777] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:28 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:28.362929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736ea31-b41c-4495-87a6-b4f09dfcc188 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:28.372361 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 891/1778] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:28 2022] POST /identity/v3/users => generated 286 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:28.373134 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d42cec-38c1-4ce2-813a-b80d594c0798 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:28.384570 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736ea31-b41c-4495-87a6-b4f09dfcc188 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:28.388600 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d42cec-38c1-4ce2-813a-b80d594c0798 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:28.394423 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45c544-f795-4161-9263-1e469cc59b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:28.395230 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45c544-f795-4161-9263-1e469cc59b6b admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:28.395935 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45c544-f795-4161-9263-1e469cc59b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:28.396872 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45c544-f795-4161-9263-1e469cc59b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:28.397389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45c544-f795-4161-9263-1e469cc59b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:28.400073 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45c544-f795-4161-9263-1e469cc59b6b admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:28.402328 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5940908-d499-4f73-afe5-6dc37d3cf763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:28.403522 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5940908-d499-4f73-afe5-6dc37d3cf763 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:28.403902 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5940908-d499-4f73-afe5-6dc37d3cf763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:28.404309 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5940908-d499-4f73-afe5-6dc37d3cf763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:28.406425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5940908-d499-4f73-afe5-6dc37d3cf763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:28.411430 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5940908-d499-4f73-afe5-6dc37d3cf763 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:28.419644 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 892/1779] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:28.425500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45c544-f795-4161-9263-1e469cc59b6b admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:28.441406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 888/1780] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:28.446715 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5940908-d499-4f73-afe5-6dc37d3cf763 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:28.455423 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45c544-f795-4161-9263-1e469cc59b6b admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:28.465518 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18079e2-705f-41fe-8f7a-b618967551e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:28.466125 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18079e2-705f-41fe-8f7a-b618967551e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:28.466412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18079e2-705f-41fe-8f7a-b618967551e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:28.468408 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18079e2-705f-41fe-8f7a-b618967551e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:28.471373 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18079e2-705f-41fe-8f7a-b618967551e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:28.472901 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5940908-d499-4f73-afe5-6dc37d3cf763 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:28.474135 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18079e2-705f-41fe-8f7a-b618967551e0 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:28.484286 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b3ea5-c6ce-4128-a3a6-8f2304554a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:28.486158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b3ea5-c6ce-4128-a3a6-8f2304554a91 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:28.486453 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b3ea5-c6ce-4128-a3a6-8f2304554a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:28.486733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b3ea5-c6ce-4128-a3a6-8f2304554a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:28.489010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b3ea5-c6ce-4128-a3a6-8f2304554a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:28.490643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b3ea5-c6ce-4128-a3a6-8f2304554a91 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:28.513335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 893/1781] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:28.516088 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18079e2-705f-41fe-8f7a-b618967551e0 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:28.536843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 889/1782] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:28.540111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b3ea5-c6ce-4128-a3a6-8f2304554a91 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:28.551187 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18079e2-705f-41fe-8f7a-b618967551e0 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:28.579683 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc36440-9766-4a6b-9ce6-a70e089229ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:28.580867 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc36440-9766-4a6b-9ce6-a70e089229ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:28.581919 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc36440-9766-4a6b-9ce6-a70e089229ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:28.582405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc36440-9766-4a6b-9ce6-a70e089229ad admin admin] PATH_INFO: `/v3/projects/7a5149b487a845ec8a15122a2531fad3/users/79db19df04eb4bd38f967015efcd843c/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:28.582643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b3ea5-c6ce-4128-a3a6-8f2304554a91 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:28.594443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aaede5-2d99-4db5-b84b-8662571396e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:28.596962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aaede5-2d99-4db5-b84b-8662571396e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:28.598736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aaede5-2d99-4db5-b84b-8662571396e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:28.599493 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aaede5-2d99-4db5-b84b-8662571396e3 admin admin] PATH_INFO: `/v3/projects/50b8259163124d649ffa6202dab55736/users/78b6d6dd87bc41b191ab96ad6fc9068b/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:28.616606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:28.620480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc36440-9766-4a6b-9ce6-a70e089229ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a5149b487a845ec8a15122a2531fad3, user_id=79db19df04eb4bd38f967015efcd843c, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:28.627027 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc36440-9766-4a6b-9ce6-a70e089229ad admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:28.636205 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aaede5-2d99-4db5-b84b-8662571396e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50b8259163124d649ffa6202dab55736, user_id=78b6d6dd87bc41b191ab96ad6fc9068b, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:28.642092 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aaede5-2d99-4db5-b84b-8662571396e3 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:28.666022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 894/1783] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:28 2022] PUT /identity/v3/projects/7a5149b487a845ec8a15122a2531fad3/users/79db19df04eb4bd38f967015efcd843c/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:28.670473 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556273f7-79c1-4413-979e-c92bd6bafa73 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:28.671945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556273f7-79c1-4413-979e-c92bd6bafa73 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:28.672233 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556273f7-79c1-4413-979e-c92bd6bafa73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:28.677813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 890/1784] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:28 2022] PUT /identity/v3/projects/50b8259163124d649ffa6202dab55736/users/78b6d6dd87bc41b191ab96ad6fc9068b/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:28.692982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd03cb-4abf-4b3a-af1d-6a82ab73be74 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:28.694419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd03cb-4abf-4b3a-af1d-6a82ab73be74 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:28.694706 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd03cb-4abf-4b3a-af1d-6a82ab73be74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:28.742725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-556273f7-79c1-4413-979e-c92bd6bafa73 None None] MFA Rules not processed for user `79db19df04eb4bd38f967015efcd843c`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:28.744014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-556273f7-79c1-4413-979e-c92bd6bafa73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:28.750429 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-02fd03cb-4abf-4b3a-af1d-6a82ab73be74 None None] MFA Rules not processed for user `78b6d6dd87bc41b191ab96ad6fc9068b`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:28.752625 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-02fd03cb-4abf-4b3a-af1d-6a82ab73be74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:28.921955 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 895/1785] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:28.930736 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556273f7-79c1-4413-979e-c92bd6bafa73 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:28.946621 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 891/1786] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:28.947893 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd03cb-4abf-4b3a-af1d-6a82ab73be74 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:28.955238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-02fd03cb-4abf-4b3a-af1d-6a82ab73be74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:29.018146 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:29.078550 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556273f7-79c1-4413-979e-c92bd6bafa73 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:29.086850 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7118303-df84-4979-bd6f-43a8717226da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:29.087525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7118303-df84-4979-bd6f-43a8717226da admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:29.087854 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7118303-df84-4979-bd6f-43a8717226da admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:29.088142 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7118303-df84-4979-bd6f-43a8717226da admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:29.088849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7118303-df84-4979-bd6f-43a8717226da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:29.096465 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7118303-df84-4979-bd6f-43a8717226da admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:29.141477 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd03cb-4abf-4b3a-af1d-6a82ab73be74 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:29.156430 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b1f72-c7f3-4b68-8134-1920543ed53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:29.157651 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b1f72-c7f3-4b68-8134-1920543ed53f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:29.157937 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b1f72-c7f3-4b68-8134-1920543ed53f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:29.158244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b1f72-c7f3-4b68-8134-1920543ed53f admin admin] PATH_INFO: `/v3/projects/4d89f821ea534751b38d212cf85490fa` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:29.160589 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b1f72-c7f3-4b68-8134-1920543ed53f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d89f821ea534751b38d212cf85490fa)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:29.166319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b1f72-c7f3-4b68-8134-1920543ed53f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:29.169845 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 896/1787] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:28 2022] POST /identity/v3/projects => generated 373 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:29.169845 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7118303-df84-4979-bd6f-43a8717226da admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:29.327885 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3a0b1f72-c7f3-4b68-8134-1920543ed53f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d89f821ea534751b38d212cf85490fa', 'request_id': 'req-3a0b1f72-c7f3-4b68-8134-1920543ed53f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:29.340048 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3a0b1f72-c7f3-4b68-8134-1920543ed53f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d89f821ea534751b38d212cf85490fa', 'request_id': 'req-3a0b1f72-c7f3-4b68-8134-1920543ed53f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:29.364442 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3a0b1f72-c7f3-4b68-8134-1920543ed53f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d89f821ea534751b38d212cf85490fa', 'request_id': 'req-3a0b1f72-c7f3-4b68-8134-1920543ed53f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:29.404117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3a0b1f72-c7f3-4b68-8134-1920543ed53f admin admin] The token cache is being invalidate because project 4d89f821ea534751b38d212cf85490fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:29.406142 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3a0b1f72-c7f3-4b68-8134-1920543ed53f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:29.407250 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3a0b1f72-c7f3-4b68-8134-1920543ed53f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:29.410714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 892/1788] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:28 2022] DELETE /identity/v3/projects/4d89f821ea534751b38d212cf85490fa => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:29.414858 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b1f72-c7f3-4b68-8134-1920543ed53f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:29.416393 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3a0b1f72-c7f3-4b68-8134-1920543ed53f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:29.429090 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7118303-df84-4979-bd6f-43a8717226da admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:29.437445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:29.596049 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b1f72-c7f3-4b68-8134-1920543ed53f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:29.603045 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0731b7-c3f8-485d-a235-feb5b838c1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:29.603822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0731b7-c3f8-485d-a235-feb5b838c1c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:29.604292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0731b7-c3f8-485d-a235-feb5b838c1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:29.604578 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0731b7-c3f8-485d-a235-feb5b838c1c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:29.605505 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0731b7-c3f8-485d-a235-feb5b838c1c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:29.607555 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0731b7-c3f8-485d-a235-feb5b838c1c6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:29.621482 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942f891-59d8-42c8-abd0-c5dde3f3bfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:29.621922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942f891-59d8-42c8-abd0-c5dde3f3bfb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:29.622212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942f891-59d8-42c8-abd0-c5dde3f3bfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:29.622488 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942f891-59d8-42c8-abd0-c5dde3f3bfb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:29.623767 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942f891-59d8-42c8-abd0-c5dde3f3bfb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:29.626784 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942f891-59d8-42c8-abd0-c5dde3f3bfb2 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:29.657787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 893/1789] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:29 2022] POST /identity/v3/users => generated 293 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:29.659284 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513418c-b020-49c7-8059-0b8dc18eb4bb None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:29.659575 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513418c-b020-49c7-8059-0b8dc18eb4bb None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:29.659874 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513418c-b020-49c7-8059-0b8dc18eb4bb None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:29.662543 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 894/1790] 10.222.0.42 () {54 vars in 912 bytes} [Mon Jun 6 10:38:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:38:29.664912 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f5f7a-0955-4d0b-a3a9-71fe65cd2c9a None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:29.666011 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f5f7a-0955-4d0b-a3a9-71fe65cd2c9a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:29.666011 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f5f7a-0955-4d0b-a3a9-71fe65cd2c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:29.688762 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 897/1791] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:29 2022] POST /identity/v3/projects => generated 387 bytes in 521 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:29.689238 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942f891-59d8-42c8-abd0-c5dde3f3bfb2 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:29.712136 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9a0f5f7a-0955-4d0b-a3a9-71fe65cd2c9a None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:29.713740 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9a0f5f7a-0955-4d0b-a3a9-71fe65cd2c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:29.861557 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942f891-59d8-42c8-abd0-c5dde3f3bfb2 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:29.875915 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802be61-ca39-46f9-9188-7e9fc6fc7c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:29.876719 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802be61-ca39-46f9-9188-7e9fc6fc7c60 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:29.876997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802be61-ca39-46f9-9188-7e9fc6fc7c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:29.877379 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802be61-ca39-46f9-9188-7e9fc6fc7c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:29.878640 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5802be61-ca39-46f9-9188-7e9fc6fc7c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:29.879721 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5802be61-ca39-46f9-9188-7e9fc6fc7c60 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:29.886831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 895/1792] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:29.888332 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db381343-79f9-47ab-be38-cef08e1418ba None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:29.888622 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db381343-79f9-47ab-be38-cef08e1418ba None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:29.889507 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db381343-79f9-47ab-be38-cef08e1418ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:29.906389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:29.929053 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 898/1793] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:29.930113 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802be61-ca39-46f9-9188-7e9fc6fc7c60 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:29.939923 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-5802be61-ca39-46f9-9188-7e9fc6fc7c60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:29.956567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-db381343-79f9-47ab-be38-cef08e1418ba None None] MFA Rules not processed for user `881069f9b40a485dac2dda684dd6cd3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:29.958293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-db381343-79f9-47ab-be38-cef08e1418ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:29.971234 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802be61-ca39-46f9-9188-7e9fc6fc7c60 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:29.982587 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866e16a-5e04-4b4c-9f39-888aa001d0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:29.985126 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866e16a-5e04-4b4c-9f39-888aa001d0c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:29.986502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866e16a-5e04-4b4c-9f39-888aa001d0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:29.986821 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866e16a-5e04-4b4c-9f39-888aa001d0c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:29.988960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866e16a-5e04-4b4c-9f39-888aa001d0c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:29.993520 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866e16a-5e04-4b4c-9f39-888aa001d0c1 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:30.056412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 899/1794] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:29 2022] POST /identity/v3/users => generated 294 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:30.057976 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099a1a8-4dc6-4d1e-8e45-2a2062914fdf None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:30.058304 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099a1a8-4dc6-4d1e-8e45-2a2062914fdf None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:30.063644 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099a1a8-4dc6-4d1e-8e45-2a2062914fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:30.104497 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 896/1795] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jun 6 10:38:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:30.106809 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db381343-79f9-47ab-be38-cef08e1418ba None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:30.109994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-d099a1a8-4dc6-4d1e-8e45-2a2062914fdf None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:30.114389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-d099a1a8-4dc6-4d1e-8e45-2a2062914fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:30.134704 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db381343-79f9-47ab-be38-cef08e1418ba None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:30.136735 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 900/1796] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:30.138049 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099a1a8-4dc6-4d1e-8e45-2a2062914fdf None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:30.147742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a916b7-46d0-4832-aabf-6cc4227c3fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:30.148760 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a916b7-46d0-4832-aabf-6cc4227c3fea admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:30.149043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a916b7-46d0-4832-aabf-6cc4227c3fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:30.150003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a916b7-46d0-4832-aabf-6cc4227c3fea admin admin] PATH_INFO: `/v3/projects/3025ef996c1b48f2b8f925dae544bfeb/users/726f8a21a9a94192a737c6ce2eb1a5cf/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:30.167144 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099a1a8-4dc6-4d1e-8e45-2a2062914fdf None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:30.178718 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40c5e6-1845-460b-ae16-122cc9406b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:30.180907 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40c5e6-1845-460b-ae16-122cc9406b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:30.181200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40c5e6-1845-460b-ae16-122cc9406b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:30.181475 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40c5e6-1845-460b-ae16-122cc9406b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:30.182264 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40c5e6-1845-460b-ae16-122cc9406b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:30.185179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40c5e6-1845-460b-ae16-122cc9406b3a admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:30.189613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a916b7-46d0-4832-aabf-6cc4227c3fea admin admin] RBAC: Authorizing `identity:create_grant(project_id=3025ef996c1b48f2b8f925dae544bfeb, user_id=726f8a21a9a94192a737c6ce2eb1a5cf, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:30.192691 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a916b7-46d0-4832-aabf-6cc4227c3fea admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:30.207758 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 901/1797] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:30.210221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820150e4-f8b2-468a-88b5-a6980c47e3d1 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:30.210582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820150e4-f8b2-468a-88b5-a6980c47e3d1 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:30.210931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820150e4-f8b2-468a-88b5-a6980c47e3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:30.223704 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 897/1798] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:30 2022] PUT /identity/v3/projects/3025ef996c1b48f2b8f925dae544bfeb/users/726f8a21a9a94192a737c6ce2eb1a5cf/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:30.226786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a916b7-46d0-4832-aabf-6cc4227c3fea admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:30.265676 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-820150e4-f8b2-468a-88b5-a6980c47e3d1 None None] MFA Rules not processed for user `ea1482ba53554f71865cea46ffbac23c`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:30.270505 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-820150e4-f8b2-468a-88b5-a6980c47e3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:30.289006 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:30.375572 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a916b7-46d0-4832-aabf-6cc4227c3fea admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:30.394274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370df94b-18ea-46ba-9177-86ee859026ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:30.395449 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370df94b-18ea-46ba-9177-86ee859026ab admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:30.396392 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370df94b-18ea-46ba-9177-86ee859026ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:30.396851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370df94b-18ea-46ba-9177-86ee859026ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:30.398096 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370df94b-18ea-46ba-9177-86ee859026ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:38:30.398306 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370df94b-18ea-46ba-9177-86ee859026ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:30.400340 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370df94b-18ea-46ba-9177-86ee859026ab admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:30.411853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 902/1799] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:30.413010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820150e4-f8b2-468a-88b5-a6980c47e3d1 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:30.414843 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-820150e4-f8b2-468a-88b5-a6980c47e3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:30.415873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 898/1800] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:38:30.416854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370df94b-18ea-46ba-9177-86ee859026ab admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:30.429210 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370df94b-18ea-46ba-9177-86ee859026ab admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:30.437238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40781777-a219-4234-b2c7-cb78a5b9bec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:30.438503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40781777-a219-4234-b2c7-cb78a5b9bec1 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:30.439037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40781777-a219-4234-b2c7-cb78a5b9bec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:30.439675 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40781777-a219-4234-b2c7-cb78a5b9bec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:30.440937 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40781777-a219-4234-b2c7-cb78a5b9bec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:30.443465 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40781777-a219-4234-b2c7-cb78a5b9bec1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:30.472607 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 899/1801] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:30.473669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40781777-a219-4234-b2c7-cb78a5b9bec1 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:30.489622 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40781777-a219-4234-b2c7-cb78a5b9bec1 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:30.497987 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34cf77-7c87-4030-81d6-fc7933b1bd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:30.500346 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34cf77-7c87-4030-81d6-fc7933b1bd2e admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:30.500811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34cf77-7c87-4030-81d6-fc7933b1bd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:30.501601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34cf77-7c87-4030-81d6-fc7933b1bd2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:30.502549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34cf77-7c87-4030-81d6-fc7933b1bd2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:30.505332 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34cf77-7c87-4030-81d6-fc7933b1bd2e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:30.519014 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 900/1802] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:30.519803 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34cf77-7c87-4030-81d6-fc7933b1bd2e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:30.547254 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34cf77-7c87-4030-81d6-fc7933b1bd2e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:30.553507 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa42f1a-36f9-4d82-b807-16d25a58447c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:30.577292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa42f1a-36f9-4d82-b807-16d25a58447c admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:30.577578 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa42f1a-36f9-4d82-b807-16d25a58447c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:30.577860 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa42f1a-36f9-4d82-b807-16d25a58447c admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:30.580809 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa42f1a-36f9-4d82-b807-16d25a58447c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:30.588970 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa42f1a-36f9-4d82-b807-16d25a58447c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:30.614729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820150e4-f8b2-468a-88b5-a6980c47e3d1 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:30.618497 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bbc9d-6223-46b9-9841-249ea3d7e4df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:30.619362 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bbc9d-6223-46b9-9841-249ea3d7e4df service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:30.619653 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bbc9d-6223-46b9-9841-249ea3d7e4df service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:30.620656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bbc9d-6223-46b9-9841-249ea3d7e4df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:30.642474 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2bbc9d-6223-46b9-9841-249ea3d7e4df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:30.643988 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2bbc9d-6223-46b9-9841-249ea3d7e4df service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:30.675362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 901/1803] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:30 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:30.676429 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa42f1a-36f9-4d82-b807-16d25a58447c admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:30.679859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 903/1804] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:30.684653 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bbc9d-6223-46b9-9841-249ea3d7e4df service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:30.727897 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bbc9d-6223-46b9-9841-249ea3d7e4df service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:30.785761 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa42f1a-36f9-4d82-b807-16d25a58447c admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:30.793873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed6f95-3c18-4bfb-9210-324b9bc5d890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:30.794991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed6f95-3c18-4bfb-9210-324b9bc5d890 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:30.795621 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed6f95-3c18-4bfb-9210-324b9bc5d890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:30.796262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed6f95-3c18-4bfb-9210-324b9bc5d890 admin admin] PATH_INFO: `/v3/projects/1e8e29f8eb004f96a5cd17a03cc27c19/users/57966351a15d443eb7fe9c46b3bea34d/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:30.820226 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed6f95-3c18-4bfb-9210-324b9bc5d890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e8e29f8eb004f96a5cd17a03cc27c19, user_id=57966351a15d443eb7fe9c46b3bea34d, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:30.829268 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed6f95-3c18-4bfb-9210-324b9bc5d890 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:30.876363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 902/1805] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:30 2022] PUT /identity/v3/projects/1e8e29f8eb004f96a5cd17a03cc27c19/users/57966351a15d443eb7fe9c46b3bea34d/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:30.877302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed6f95-3c18-4bfb-9210-324b9bc5d890 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:30.886611 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d4f045-d332-4faf-b9b0-f8eabed51c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:30.887527 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d4f045-d332-4faf-b9b0-f8eabed51c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:30.890045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d4f045-d332-4faf-b9b0-f8eabed51c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:30.890470 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d4f045-d332-4faf-b9b0-f8eabed51c4a admin admin] PATH_INFO: `/v3/projects/3025ef996c1b48f2b8f925dae544bfeb/users/726f8a21a9a94192a737c6ce2eb1a5cf/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:30.896933 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d4f045-d332-4faf-b9b0-f8eabed51c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3025ef996c1b48f2b8f925dae544bfeb, user_id=726f8a21a9a94192a737c6ce2eb1a5cf, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:30.899773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d4f045-d332-4faf-b9b0-f8eabed51c4a admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:30.925695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 904/1806] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:30 2022] PUT /identity/v3/projects/3025ef996c1b48f2b8f925dae544bfeb/users/726f8a21a9a94192a737c6ce2eb1a5cf/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:30.928675 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15830c81-e873-42ca-a98b-ce0ad6ce5356 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:30.929242 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15830c81-e873-42ca-a98b-ce0ad6ce5356 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:30.929852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15830c81-e873-42ca-a98b-ce0ad6ce5356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:30.939689 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:30.983467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-15830c81-e873-42ca-a98b-ce0ad6ce5356 None None] MFA Rules not processed for user `57966351a15d443eb7fe9c46b3bea34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:30.983859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:30.986801 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-15830c81-e873-42ca-a98b-ce0ad6ce5356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:30.991756 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed6f95-3c18-4bfb-9210-324b9bc5d890 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:31.100154 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 905/1807] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:31.100154 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994f470-8c5b-4a0d-87de-1ab083cb8730 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:31.100154 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994f470-8c5b-4a0d-87de-1ab083cb8730 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:31.100154 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994f470-8c5b-4a0d-87de-1ab083cb8730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:31.106991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304d02d-5867-4610-929d-369c0d063f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:31.108131 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304d02d-5867-4610-929d-369c0d063f53 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:31.108644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304d02d-5867-4610-929d-369c0d063f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:31.109774 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304d02d-5867-4610-929d-369c0d063f53 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:31.110764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304d02d-5867-4610-929d-369c0d063f53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:31.113669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304d02d-5867-4610-929d-369c0d063f53 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:31.150815 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-2994f470-8c5b-4a0d-87de-1ab083cb8730 None None] MFA Rules not processed for user `c47d8ba602c54febac5f6203f9517351`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:31.152149 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-2994f470-8c5b-4a0d-87de-1ab083cb8730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:31.158583 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 903/1808] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:30 2022] POST /identity/v3/users => generated 294 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:31.161336 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b05a3-d889-4bff-b3a1-3a9e146405cf None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:31.164645 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b05a3-d889-4bff-b3a1-3a9e146405cf None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:31.165617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b05a3-d889-4bff-b3a1-3a9e146405cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:31.218687 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9b5b05a3-d889-4bff-b3a1-3a9e146405cf None None] MFA Rules not processed for user `726f8a21a9a94192a737c6ce2eb1a5cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:31.220332 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9b5b05a3-d889-4bff-b3a1-3a9e146405cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:31.291569 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 906/1809] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:31.291569 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994f470-8c5b-4a0d-87de-1ab083cb8730 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:31.315996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994f470-8c5b-4a0d-87de-1ab083cb8730 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:31.326584 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6601c4-0211-441a-af5a-cccbc7a1c61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:31.327364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6601c4-0211-441a-af5a-cccbc7a1c61c admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:31.327647 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6601c4-0211-441a-af5a-cccbc7a1c61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:31.327949 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6601c4-0211-441a-af5a-cccbc7a1c61c admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:31.328658 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6601c4-0211-441a-af5a-cccbc7a1c61c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:31.336127 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6601c4-0211-441a-af5a-cccbc7a1c61c admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:31.398654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 907/1810] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:31 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:31.399687 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6601c4-0211-441a-af5a-cccbc7a1c61c admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:31.427412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 904/1811] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:31.427412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b05a3-d889-4bff-b3a1-3a9e146405cf None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:31.429308 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9b5b05a3-d889-4bff-b3a1-3a9e146405cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:31.457936 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b05a3-d889-4bff-b3a1-3a9e146405cf None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:31.495602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6601c4-0211-441a-af5a-cccbc7a1c61c admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:31.501311 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52751e36-f496-430f-a347-775111a0acb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:31.502556 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52751e36-f496-430f-a347-775111a0acb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:31.502842 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52751e36-f496-430f-a347-775111a0acb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:31.503112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52751e36-f496-430f-a347-775111a0acb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:31.504005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52751e36-f496-430f-a347-775111a0acb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:31.505971 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52751e36-f496-430f-a347-775111a0acb1 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:31.508764 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:31.519465 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 908/1812] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:31.520461 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52751e36-f496-430f-a347-775111a0acb1 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:31.546278 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52751e36-f496-430f-a347-775111a0acb1 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:31.552359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718072d-0617-4bc9-a52a-59fb28d0036e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:31.553276 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718072d-0617-4bc9-a52a-59fb28d0036e admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:31.553614 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718072d-0617-4bc9-a52a-59fb28d0036e admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:31.553885 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718072d-0617-4bc9-a52a-59fb28d0036e admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:31.554629 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718072d-0617-4bc9-a52a-59fb28d0036e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:31.557046 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718072d-0617-4bc9-a52a-59fb28d0036e admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:31.612782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 909/1813] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:31 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:31.613758 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718072d-0617-4bc9-a52a-59fb28d0036e admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:31.614841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02d629-5a04-4f3f-abc7-a015950d4f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:31.615862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-0718072d-0617-4bc9-a52a-59fb28d0036e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:31.616532 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02d629-5a04-4f3f-abc7-a015950d4f82 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:31.616984 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02d629-5a04-4f3f-abc7-a015950d4f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:31.617419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02d629-5a04-4f3f-abc7-a015950d4f82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:31.618269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02d629-5a04-4f3f-abc7-a015950d4f82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:31.625088 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02d629-5a04-4f3f-abc7-a015950d4f82 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:31.693010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 905/1814] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:31 2022] POST /identity/v3/projects => generated 371 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:31.693010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02d629-5a04-4f3f-abc7-a015950d4f82 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:31.695523 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:31.709216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718072d-0617-4bc9-a52a-59fb28d0036e admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:31.774352 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86061a8-121c-4bc8-ae91-131381706b27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:31.775000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86061a8-121c-4bc8-ae91-131381706b27 service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:31.775281 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86061a8-121c-4bc8-ae91-131381706b27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:31.775558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86061a8-121c-4bc8-ae91-131381706b27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:31.798909 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02d629-5a04-4f3f-abc7-a015950d4f82 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:31.805737 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0bfaca-d928-428a-be80-9da8f69ba3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:31.806733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0bfaca-d928-428a-be80-9da8f69ba3df admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:31.808077 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0bfaca-d928-428a-be80-9da8f69ba3df admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:31.808570 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0bfaca-d928-428a-be80-9da8f69ba3df admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:31.809648 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0bfaca-d928-428a-be80-9da8f69ba3df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:31.812524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0bfaca-d928-428a-be80-9da8f69ba3df admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:31.829399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 906/1815] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:31.830977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0bfaca-d928-428a-be80-9da8f69ba3df admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:31.845549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0bfaca-d928-428a-be80-9da8f69ba3df admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:31.853575 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cf442-3e55-43da-9b18-ce0d2631c417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:31.854647 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cf442-3e55-43da-9b18-ce0d2631c417 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:31.855655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cf442-3e55-43da-9b18-ce0d2631c417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:31.856157 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cf442-3e55-43da-9b18-ce0d2631c417 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:31.858324 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cf442-3e55-43da-9b18-ce0d2631c417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:31.861073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cf442-3e55-43da-9b18-ce0d2631c417 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:31.875748 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 907/1816] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:31.877293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cf442-3e55-43da-9b18-ce0d2631c417 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:31.890152 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cf442-3e55-43da-9b18-ce0d2631c417 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:31.897759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e9925-cca7-4c27-9980-4020e3ebb3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:31.899041 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e9925-cca7-4c27-9980-4020e3ebb3b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:31.899671 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e9925-cca7-4c27-9980-4020e3ebb3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:31.900184 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e9925-cca7-4c27-9980-4020e3ebb3b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:31.901088 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e9925-cca7-4c27-9980-4020e3ebb3b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:31.905604 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e9925-cca7-4c27-9980-4020e3ebb3b3 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:31.917536 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86061a8-121c-4bc8-ae91-131381706b27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:31.919661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86061a8-121c-4bc8-ae91-131381706b27 service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:31.937745 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 910/1817] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:31 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:31.938260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86061a8-121c-4bc8-ae91-131381706b27 service cinder] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:31.972347 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86061a8-121c-4bc8-ae91-131381706b27 service cinder] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:31.980169 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 908/1818] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:31 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:31.982653 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e9925-cca7-4c27-9980-4020e3ebb3b3 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:31.985572 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d6662-8e3f-4e9c-baed-63b1c5926230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:31.988785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d6662-8e3f-4e9c-baed-63b1c5926230 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:31.989332 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d6662-8e3f-4e9c-baed-63b1c5926230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:31.989735 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d6662-8e3f-4e9c-baed-63b1c5926230 admin admin] PATH_INFO: `/v3/projects/1ae2319579e04c66a683fe6c45053ade/users/b645b59876e84b31891db0eef0a06674/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:32.023384 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e9925-cca7-4c27-9980-4020e3ebb3b3 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:32.028874 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f00274-a897-41b6-b1b7-899f4d0bd36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:32.030412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f00274-a897-41b6-b1b7-899f4d0bd36a admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:32.030935 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f00274-a897-41b6-b1b7-899f4d0bd36a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:32.031339 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f00274-a897-41b6-b1b7-899f4d0bd36a admin admin] PATH_INFO: `/v3/projects/c82db829db6144e9a93adaa8010e0854/users/7217444e28154916acb4a802eb4e9704/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:32.048869 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d6662-8e3f-4e9c-baed-63b1c5926230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae2319579e04c66a683fe6c45053ade, user_id=b645b59876e84b31891db0eef0a06674, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:32.053656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f00274-a897-41b6-b1b7-899f4d0bd36a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c82db829db6144e9a93adaa8010e0854, user_id=7217444e28154916acb4a802eb4e9704, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:32.056937 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f00274-a897-41b6-b1b7-899f4d0bd36a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:32.057697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d6662-8e3f-4e9c-baed-63b1c5926230 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:32.077213 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 909/1819] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:31 2022] PUT /identity/v3/projects/c82db829db6144e9a93adaa8010e0854/users/7217444e28154916acb4a802eb4e9704/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:32.078432 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f00274-a897-41b6-b1b7-899f4d0bd36a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:32.101660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 911/1820] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:31 2022] PUT /identity/v3/projects/1ae2319579e04c66a683fe6c45053ade/users/b645b59876e84b31891db0eef0a06674/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:32.105512 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d6662-8e3f-4e9c-baed-63b1c5926230 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:32.169867 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:32.213358 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f00274-a897-41b6-b1b7-899f4d0bd36a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:32.256603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d6662-8e3f-4e9c-baed-63b1c5926230 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:32.289673 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d6ad4e-3463-43f1-bf64-4493686e0dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:32.290457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6ad4e-3463-43f1-bf64-4493686e0dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:32.290859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6ad4e-3463-43f1-bf64-4493686e0dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:32.295711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6ad4e-3463-43f1-bf64-4493686e0dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:32.296961 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d6ad4e-3463-43f1-bf64-4493686e0dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:32.299317 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d6ad4e-3463-43f1-bf64-4493686e0dc0 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:32.320882 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 912/1821] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:32.322850 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca677c1-fd4a-4b14-9ba5-7edcea324d8b None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:32.323105 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca677c1-fd4a-4b14-9ba5-7edcea324d8b None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:32.323400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca677c1-fd4a-4b14-9ba5-7edcea324d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:32.331368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:32.351683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0300c79-22d2-44b9-bcdc-4bc64d8cb762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:32.352498 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0300c79-22d2-44b9-bcdc-4bc64d8cb762 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:32.352788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0300c79-22d2-44b9-bcdc-4bc64d8cb762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:32.353065 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0300c79-22d2-44b9-bcdc-4bc64d8cb762 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:32.353812 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0300c79-22d2-44b9-bcdc-4bc64d8cb762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:32.360499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0300c79-22d2-44b9-bcdc-4bc64d8cb762 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:32.382747 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-eca677c1-fd4a-4b14-9ba5-7edcea324d8b None None] MFA Rules not processed for user `b645b59876e84b31891db0eef0a06674`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:32.384100 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-eca677c1-fd4a-4b14-9ba5-7edcea324d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:32.405317 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 910/1822] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:32.406299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0300c79-22d2-44b9-bcdc-4bc64d8cb762 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:32.532652 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0300c79-22d2-44b9-bcdc-4bc64d8cb762 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:32.542334 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b53c21-54cf-4faf-8491-2cfee93aad50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:32.543018 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b53c21-54cf-4faf-8491-2cfee93aad50 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:32.543303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b53c21-54cf-4faf-8491-2cfee93aad50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:32.543576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b53c21-54cf-4faf-8491-2cfee93aad50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:32.548053 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-90b53c21-54cf-4faf-8491-2cfee93aad50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:32.575388 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 913/1823] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:32.576449 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca677c1-fd4a-4b14-9ba5-7edcea324d8b None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:32.601668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca677c1-fd4a-4b14-9ba5-7edcea324d8b None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:32.608309 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a947e8-0118-419d-ae58-ba8263b67f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:32.610878 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a947e8-0118-419d-ae58-ba8263b67f24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:32.611270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a947e8-0118-419d-ae58-ba8263b67f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:32.611652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a947e8-0118-419d-ae58-ba8263b67f24 admin admin] PATH_INFO: `/v3/projects/c82db829db6144e9a93adaa8010e0854/users/7217444e28154916acb4a802eb4e9704/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:32.624923 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a947e8-0118-419d-ae58-ba8263b67f24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c82db829db6144e9a93adaa8010e0854, user_id=7217444e28154916acb4a802eb4e9704, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:32.628299 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a947e8-0118-419d-ae58-ba8263b67f24 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:32.655891 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 914/1824] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:32 2022] PUT /identity/v3/projects/c82db829db6144e9a93adaa8010e0854/users/7217444e28154916acb4a802eb4e9704/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:32.657168 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a947e8-0118-419d-ae58-ba8263b67f24 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:32.780482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b53c21-54cf-4faf-8491-2cfee93aad50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:32.782090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b53c21-54cf-4faf-8491-2cfee93aad50 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:32.792728 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 911/1825] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:32.793581 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b53c21-54cf-4faf-8491-2cfee93aad50 service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:32.805496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b53c21-54cf-4faf-8491-2cfee93aad50 service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:32.862901 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a947e8-0118-419d-ae58-ba8263b67f24 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:32.881445 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b73b39-ffc7-4760-8b33-693b1ff5a86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:32.885876 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b73b39-ffc7-4760-8b33-693b1ff5a86d admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:32.886741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b73b39-ffc7-4760-8b33-693b1ff5a86d admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:32.887137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b73b39-ffc7-4760-8b33-693b1ff5a86d admin admin] PATH_INFO: `/v3/projects/9afffa6608c94d81a94cfc3a7c452a7c/users/26243b992121478595db9a6de7950e9a/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:32.900438 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393597c-4369-40fa-961b-73147a91a146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:32.901118 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393597c-4369-40fa-961b-73147a91a146 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:32.903706 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393597c-4369-40fa-961b-73147a91a146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:32.906086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393597c-4369-40fa-961b-73147a91a146 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:32.907377 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b73b39-ffc7-4760-8b33-693b1ff5a86d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9afffa6608c94d81a94cfc3a7c452a7c, user_id=26243b992121478595db9a6de7950e9a, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:32.908516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393597c-4369-40fa-961b-73147a91a146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:32.910440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b73b39-ffc7-4760-8b33-693b1ff5a86d admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:32.915116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393597c-4369-40fa-961b-73147a91a146 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:32.932909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 915/1826] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:32 2022] PUT /identity/v3/projects/9afffa6608c94d81a94cfc3a7c452a7c/users/26243b992121478595db9a6de7950e9a/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:32.934549 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ba4de-c9ef-4007-900d-fb1786e63464 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:32.935104 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ba4de-c9ef-4007-900d-fb1786e63464 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:32.936448 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ba4de-c9ef-4007-900d-fb1786e63464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:32.970276 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-b70ba4de-c9ef-4007-900d-fb1786e63464 None None] MFA Rules not processed for user `7217444e28154916acb4a802eb4e9704`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:32.972290 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-b70ba4de-c9ef-4007-900d-fb1786e63464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:32.981469 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:33.014828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 912/1827] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:32 2022] POST /identity/v3/projects => generated 387 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:33.018873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393597c-4369-40fa-961b-73147a91a146 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:33.108072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:33.149746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 916/1828] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:33.155033 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ba4de-c9ef-4007-900d-fb1786e63464 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:33.206175 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393597c-4369-40fa-961b-73147a91a146 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:33.224222 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3680ff-938e-46e1-89e2-e8aa3fa298b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:33.224917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3680ff-938e-46e1-89e2-e8aa3fa298b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:33.225208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3680ff-938e-46e1-89e2-e8aa3fa298b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:33.225485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3680ff-938e-46e1-89e2-e8aa3fa298b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:33.230990 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3680ff-938e-46e1-89e2-e8aa3fa298b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:33.239833 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3680ff-938e-46e1-89e2-e8aa3fa298b9 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:33.242718 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ba4de-c9ef-4007-900d-fb1786e63464 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:33.265336 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc363329-5a90-481a-8859-5d1a5aa0403f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:33.266126 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc363329-5a90-481a-8859-5d1a5aa0403f admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:33.266406 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc363329-5a90-481a-8859-5d1a5aa0403f admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:33.267134 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc363329-5a90-481a-8859-5d1a5aa0403f admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:33.267667 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc363329-5a90-481a-8859-5d1a5aa0403f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:33.269969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc363329-5a90-481a-8859-5d1a5aa0403f admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:33.292369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 913/1829] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:33.295205 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3680ff-938e-46e1-89e2-e8aa3fa298b9 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:33.302699 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ee3680ff-938e-46e1-89e2-e8aa3fa298b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:33.320228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 917/1830] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:33 2022] POST /identity/v3/users => generated 300 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:33.321191 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc363329-5a90-481a-8859-5d1a5aa0403f admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:33.360096 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc363329-5a90-481a-8859-5d1a5aa0403f admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:33.382323 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f85333-bef2-454a-83bc-0c9103a009ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:33.392475 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f85333-bef2-454a-83bc-0c9103a009ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:33.392774 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f85333-bef2-454a-83bc-0c9103a009ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:33.393061 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f85333-bef2-454a-83bc-0c9103a009ff admin admin] PATH_INFO: `/v3/projects/9afffa6608c94d81a94cfc3a7c452a7c/users/26243b992121478595db9a6de7950e9a/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:33.399296 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f85333-bef2-454a-83bc-0c9103a009ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=9afffa6608c94d81a94cfc3a7c452a7c, user_id=26243b992121478595db9a6de7950e9a, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:33.408823 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f85333-bef2-454a-83bc-0c9103a009ff admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:33.443356 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 918/1831] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:33 2022] PUT /identity/v3/projects/9afffa6608c94d81a94cfc3a7c452a7c/users/26243b992121478595db9a6de7950e9a/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:33.444538 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f85333-bef2-454a-83bc-0c9103a009ff admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:33.484468 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3680ff-938e-46e1-89e2-e8aa3fa298b9 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:33.626926 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4795e8-63aa-45eb-9ee0-ea5cc0cefdaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:33.628172 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4795e8-63aa-45eb-9ee0-ea5cc0cefdaf service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:33.628988 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4795e8-63aa-45eb-9ee0-ea5cc0cefdaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:33.629450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4795e8-63aa-45eb-9ee0-ea5cc0cefdaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:33.640436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f85333-bef2-454a-83bc-0c9103a009ff admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:33.655227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec955b-ffae-4d2b-94b5-49d2fa8ad739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:33.657808 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec955b-ffae-4d2b-94b5-49d2fa8ad739 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:33.658114 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec955b-ffae-4d2b-94b5-49d2fa8ad739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:33.660397 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec955b-ffae-4d2b-94b5-49d2fa8ad739 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:33.666812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec955b-ffae-4d2b-94b5-49d2fa8ad739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:33.673725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec955b-ffae-4d2b-94b5-49d2fa8ad739 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:33.706185 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 919/1832] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:33.708546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec955b-ffae-4d2b-94b5-49d2fa8ad739 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:33.719101 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4795e8-63aa-45eb-9ee0-ea5cc0cefdaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:33.720610 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4795e8-63aa-45eb-9ee0-ea5cc0cefdaf service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:33.745206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec955b-ffae-4d2b-94b5-49d2fa8ad739 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:33.761826 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11ee89-1db3-47aa-aaa0-bce03512840d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:33.765362 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11ee89-1db3-47aa-aaa0-bce03512840d admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:33.765654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11ee89-1db3-47aa-aaa0-bce03512840d admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:33.765945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11ee89-1db3-47aa-aaa0-bce03512840d admin admin] PATH_INFO: `/v3/projects/e1f470a894094b47926509bd7b9a8b65/users/3a1314804aad4d0f80f403e871b2861d/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:33.789078 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 914/1833] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:33 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:33.819178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:33.827240 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f11ee89-1db3-47aa-aaa0-bce03512840d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1f470a894094b47926509bd7b9a8b65, user_id=3a1314804aad4d0f80f403e871b2861d, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:33.839682 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f11ee89-1db3-47aa-aaa0-bce03512840d admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:33.880087 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 920/1834] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:33 2022] PUT /identity/v3/projects/e1f470a894094b47926509bd7b9a8b65/users/3a1314804aad4d0f80f403e871b2861d/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:33.881820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4795e8-63aa-45eb-9ee0-ea5cc0cefdaf service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:33.897656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4795e8-63aa-45eb-9ee0-ea5cc0cefdaf service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:33.935817 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:33.991748 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064361fa-c21a-4b08-a7d9-ab4fe3e5e671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:33.992720 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064361fa-c21a-4b08-a7d9-ab4fe3e5e671 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:33.993156 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064361fa-c21a-4b08-a7d9-ab4fe3e5e671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:33.993632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064361fa-c21a-4b08-a7d9-ab4fe3e5e671 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:33.994531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064361fa-c21a-4b08-a7d9-ab4fe3e5e671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:33.998133 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064361fa-c21a-4b08-a7d9-ab4fe3e5e671 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:34.015283 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 915/1835] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:34.017621 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11ee89-1db3-47aa-aaa0-bce03512840d admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:34.034008 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11ee89-1db3-47aa-aaa0-bce03512840d admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:34.039950 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95402269-187c-4979-b309-47d9d70b29a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:34.041076 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95402269-187c-4979-b309-47d9d70b29a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:34.041660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95402269-187c-4979-b309-47d9d70b29a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:34.042079 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95402269-187c-4979-b309-47d9d70b29a5 admin admin] PATH_INFO: `/v3/projects/e1f470a894094b47926509bd7b9a8b65/users/3a1314804aad4d0f80f403e871b2861d/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:34.062035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95402269-187c-4979-b309-47d9d70b29a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1f470a894094b47926509bd7b9a8b65, user_id=3a1314804aad4d0f80f403e871b2861d, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:34.073339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95402269-187c-4979-b309-47d9d70b29a5 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:34.112123 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 921/1836] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:34 2022] PUT /identity/v3/projects/e1f470a894094b47926509bd7b9a8b65/users/3a1314804aad4d0f80f403e871b2861d/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:34.114716 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fd52c-149a-48c3-ad2d-ac2dafe17f93 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:34.115169 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fd52c-149a-48c3-ad2d-ac2dafe17f93 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:34.115616 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fd52c-149a-48c3-ad2d-ac2dafe17f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:34.161082 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-299fd52c-149a-48c3-ad2d-ac2dafe17f93 None None] MFA Rules not processed for user `3a1314804aad4d0f80f403e871b2861d`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:34.162915 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-299fd52c-149a-48c3-ad2d-ac2dafe17f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:34.433820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 916/1837] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:34.542694 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95402269-187c-4979-b309-47d9d70b29a5 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:34.740537 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95402269-187c-4979-b309-47d9d70b29a5 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:34.745637 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23cc1b5-1236-4460-8478-d3a31ebe8b44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:34.746460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23cc1b5-1236-4460-8478-d3a31ebe8b44 service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:34.746609 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23cc1b5-1236-4460-8478-d3a31ebe8b44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:34.746844 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23cc1b5-1236-4460-8478-d3a31ebe8b44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:34.771562 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23cc1b5-1236-4460-8478-d3a31ebe8b44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:34.773093 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23cc1b5-1236-4460-8478-d3a31ebe8b44 service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:34.799686 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 922/1838] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:35.143420 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fd52c-149a-48c3-ad2d-ac2dafe17f93 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:35.178367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fd52c-149a-48c3-ad2d-ac2dafe17f93 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:35.191445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f13a8e-87f8-405e-845b-11d4e6580ffe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:35.192390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f13a8e-87f8-405e-845b-11d4e6580ffe service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:35.192836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f13a8e-87f8-405e-845b-11d4e6580ffe service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:35.193275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f13a8e-87f8-405e-845b-11d4e6580ffe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:35.197948 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-25f13a8e-87f8-405e-845b-11d4e6580ffe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:35.338229 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f13a8e-87f8-405e-845b-11d4e6580ffe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:35.340990 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f13a8e-87f8-405e-845b-11d4e6580ffe service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:35.357090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 917/1839] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:35.678037 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5dbc0-2c07-42c0-92e4-8f044f5ba0eb None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:35.679282 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5dbc0-2c07-42c0-92e4-8f044f5ba0eb None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:35.679724 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5dbc0-2c07-42c0-92e4-8f044f5ba0eb None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:35.686472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 923/1840] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jun 6 10:38:35 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:38:35.698156 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9102f-3137-4b9e-b637-c4bd9c03c8c2 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:35.698624 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9102f-3137-4b9e-b637-c4bd9c03c8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:35.698939 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9102f-3137-4b9e-b637-c4bd9c03c8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:35.769736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9af9102f-3137-4b9e-b637-c4bd9c03c8c2 None None] MFA Rules not processed for user `ed8a40e4f1034a8e99d0eb505e448db9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:35.772794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9af9102f-3137-4b9e-b637-c4bd9c03c8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:35.821964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5dbc0-2c07-42c0-92e4-8f044f5ba0eb None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:35.984711 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:36.039766 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5dbc0-2c07-42c0-92e4-8f044f5ba0eb None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:36.046548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e49798-fd77-4b2c-8b1e-716ff9e65984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:36.047701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e49798-fd77-4b2c-8b1e-716ff9e65984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:36.048039 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e49798-fd77-4b2c-8b1e-716ff9e65984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:36.048981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e49798-fd77-4b2c-8b1e-716ff9e65984 admin admin] PATH_INFO: `/v3/users/b645b59876e84b31891db0eef0a06674` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:36.049965 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e49798-fd77-4b2c-8b1e-716ff9e65984 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b645b59876e84b31891db0eef0a06674)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:36.054402 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e49798-fd77-4b2c-8b1e-716ff9e65984 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:36.119038 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 918/1841] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jun 6 10:38:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:36.137572 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9102f-3137-4b9e-b637-c4bd9c03c8c2 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:36.140417 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9af9102f-3137-4b9e-b637-c4bd9c03c8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:36.161656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9102f-3137-4b9e-b637-c4bd9c03c8c2 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:36.169405 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1c2d4-af1a-4d16-9527-55362aef8abd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:36.170842 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1c2d4-af1a-4d16-9527-55362aef8abd service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:36.171289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1c2d4-af1a-4d16-9527-55362aef8abd service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:36.171950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1c2d4-af1a-4d16-9527-55362aef8abd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:36.186219 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1c2d4-af1a-4d16-9527-55362aef8abd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:36.188374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1c2d4-af1a-4d16-9527-55362aef8abd service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:36.193970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-09e49798-fd77-4b2c-8b1e-716ff9e65984 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b645b59876e84b31891db0eef0a06674'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:36.202839 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 919/1842] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:36.214794 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-09e49798-fd77-4b2c-8b1e-716ff9e65984 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b645b59876e84b31891db0eef0a06674'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:36.247183 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-09e49798-fd77-4b2c-8b1e-716ff9e65984 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b645b59876e84b31891db0eef0a06674'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:36.248273 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-09e49798-fd77-4b2c-8b1e-716ff9e65984 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b645b59876e84b31891db0eef0a06674'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:36.271031 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-09e49798-fd77-4b2c-8b1e-716ff9e65984 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b645b59876e84b31891db0eef0a06674'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:36.276762 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:36.294099 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 924/1843] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:35 2022] DELETE /identity/v3/users/b645b59876e84b31891db0eef0a06674 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:36.295837 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6faf7-da79-4161-9b8d-99e42ca29448 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:36.296101 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6faf7-da79-4161-9b8d-99e42ca29448 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:36.296500 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6faf7-da79-4161-9b8d-99e42ca29448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:36.340614 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9ee6faf7-da79-4161-9b8d-99e42ca29448 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:36.349247 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9ee6faf7-da79-4161-9b8d-99e42ca29448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:36.391195 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6edbe-7bd1-448c-973a-e24c85923505 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:36.391195 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6edbe-7bd1-448c-973a-e24c85923505 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:36.391195 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6edbe-7bd1-448c-973a-e24c85923505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:36.442912 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-82e6edbe-7bd1-448c-973a-e24c85923505 None None] MFA Rules not processed for user `26243b992121478595db9a6de7950e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:36.445049 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-82e6edbe-7bd1-448c-973a-e24c85923505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:36.536515 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 920/1844] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:36.579650 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 925/1845] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:36 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:36.598229 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6faf7-da79-4161-9b8d-99e42ca29448 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:36.600118 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9ee6faf7-da79-4161-9b8d-99e42ca29448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:36.687776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1deb0b-3b1e-4418-9796-76c8f8771275 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:36.688163 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1deb0b-3b1e-4418-9796-76c8f8771275 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:36.688448 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1deb0b-3b1e-4418-9796-76c8f8771275 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:36.690211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 926/1846] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jun 6 10:38:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:38:36.712429 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6faf7-da79-4161-9b8d-99e42ca29448 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:36.718670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089e551-acd7-4183-b2ac-ebda99942f26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:36.719360 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089e551-acd7-4183-b2ac-ebda99942f26 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:36.719641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089e551-acd7-4183-b2ac-ebda99942f26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:36.719983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089e551-acd7-4183-b2ac-ebda99942f26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:36.723482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e089e551-acd7-4183-b2ac-ebda99942f26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:36.725894 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1690c-b962-440c-adad-3d80cbb46585 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:36.726084 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1690c-b962-440c-adad-3d80cbb46585 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:36.726334 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1690c-b962-440c-adad-3d80cbb46585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:36.753617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-d7d1690c-b962-440c-adad-3d80cbb46585 None None] MFA Rules not processed for user `ed8a40e4f1034a8e99d0eb505e448db9`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:36.754933 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-d7d1690c-b962-440c-adad-3d80cbb46585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:36.848827 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 927/1847] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jun 6 10:38:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:36.870144 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089e551-acd7-4183-b2ac-ebda99942f26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:36.872542 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089e551-acd7-4183-b2ac-ebda99942f26 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:36.892401 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 921/1848] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:36.908650 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1690c-b962-440c-adad-3d80cbb46585 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:36.910237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-d7d1690c-b962-440c-adad-3d80cbb46585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:37.038380 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1690c-b962-440c-adad-3d80cbb46585 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:37.049505 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b972329-812b-4d53-b467-8faa98a01000 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:37.050229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b972329-812b-4d53-b467-8faa98a01000 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:37.050515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b972329-812b-4d53-b467-8faa98a01000 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:37.050789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b972329-812b-4d53-b467-8faa98a01000 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:37.073322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b972329-812b-4d53-b467-8faa98a01000 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:37.074787 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b972329-812b-4d53-b467-8faa98a01000 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:37.102537 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 928/1849] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:37.648848 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089e551-acd7-4183-b2ac-ebda99942f26 service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:37.650485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e089e551-acd7-4183-b2ac-ebda99942f26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:37.677818 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089e551-acd7-4183-b2ac-ebda99942f26 service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:37.683436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99752ccd-011c-4982-92bc-949ba47e177f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:37.684268 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99752ccd-011c-4982-92bc-949ba47e177f service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:37.685239 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99752ccd-011c-4982-92bc-949ba47e177f service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:37.686229 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99752ccd-011c-4982-92bc-949ba47e177f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:37.696845 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99752ccd-011c-4982-92bc-949ba47e177f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:37.698418 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99752ccd-011c-4982-92bc-949ba47e177f service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:37.709942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 922/1850] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:38.047462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b972329-812b-4d53-b467-8faa98a01000 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:38.049600 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-2b972329-812b-4d53-b467-8faa98a01000 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:38.066563 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b972329-812b-4d53-b467-8faa98a01000 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:38.076701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf80d7a-2eee-41c5-854e-2aa02d4185f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:38.078261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf80d7a-2eee-41c5-854e-2aa02d4185f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:38.078542 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf80d7a-2eee-41c5-854e-2aa02d4185f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:38.079501 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf80d7a-2eee-41c5-854e-2aa02d4185f9 admin admin] PATH_INFO: `/v3/projects/1ae2319579e04c66a683fe6c45053ade` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:38.081280 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf80d7a-2eee-41c5-854e-2aa02d4185f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ae2319579e04c66a683fe6c45053ade)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:38.083231 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf80d7a-2eee-41c5-854e-2aa02d4185f9 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:38.100947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:38.133702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-cdf80d7a-2eee-41c5-854e-2aa02d4185f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ae2319579e04c66a683fe6c45053ade', 'request_id': 'req-cdf80d7a-2eee-41c5-854e-2aa02d4185f9'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:38.139935 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-cdf80d7a-2eee-41c5-854e-2aa02d4185f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ae2319579e04c66a683fe6c45053ade', 'request_id': 'req-cdf80d7a-2eee-41c5-854e-2aa02d4185f9'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:38.147855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-cdf80d7a-2eee-41c5-854e-2aa02d4185f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ae2319579e04c66a683fe6c45053ade', 'request_id': 'req-cdf80d7a-2eee-41c5-854e-2aa02d4185f9'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:38.161385 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-cdf80d7a-2eee-41c5-854e-2aa02d4185f9 admin admin] The token cache is being invalidate because project 1ae2319579e04c66a683fe6c45053ade was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:38.161725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-cdf80d7a-2eee-41c5-854e-2aa02d4185f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:38.162487 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-cdf80d7a-2eee-41c5-854e-2aa02d4185f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:38.164600 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 929/1851] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:38 2022] DELETE /identity/v3/projects/1ae2319579e04c66a683fe6c45053ade => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:38.168964 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99752ccd-011c-4982-92bc-949ba47e177f service nova] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:38.185494 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99752ccd-011c-4982-92bc-949ba47e177f service nova] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:38.198431 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:38.260425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba3866-1d8f-4576-b3f0-aeef9c6fd0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:38.261505 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba3866-1d8f-4576-b3f0-aeef9c6fd0d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:38.261744 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba3866-1d8f-4576-b3f0-aeef9c6fd0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:38.262027 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba3866-1d8f-4576-b3f0-aeef9c6fd0d2 admin admin] PATH_INFO: `/v3/users/3a1314804aad4d0f80f403e871b2861d` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:38.264775 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba3866-1d8f-4576-b3f0-aeef9c6fd0d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a1314804aad4d0f80f403e871b2861d)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:38.267854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba3866-1d8f-4576-b3f0-aeef9c6fd0d2 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:38.341265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-9cba3866-1d8f-4576-b3f0-aeef9c6fd0d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a1314804aad4d0f80f403e871b2861d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:38.350127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-9cba3866-1d8f-4576-b3f0-aeef9c6fd0d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a1314804aad4d0f80f403e871b2861d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:38.350871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-9cba3866-1d8f-4576-b3f0-aeef9c6fd0d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a1314804aad4d0f80f403e871b2861d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:38.351496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-9cba3866-1d8f-4576-b3f0-aeef9c6fd0d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a1314804aad4d0f80f403e871b2861d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:38.366173 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-9cba3866-1d8f-4576-b3f0-aeef9c6fd0d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a1314804aad4d0f80f403e871b2861d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:38.403097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 923/1852] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:38 2022] DELETE /identity/v3/users/3a1314804aad4d0f80f403e871b2861d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:38.790723 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf80d7a-2eee-41c5-854e-2aa02d4185f9 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:38.793045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-cdf80d7a-2eee-41c5-854e-2aa02d4185f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:38.811448 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf80d7a-2eee-41c5-854e-2aa02d4185f9 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:38.875943 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0b9e2-a554-4f9a-9036-b6c2412f2740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:38.877102 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0b9e2-a554-4f9a-9036-b6c2412f2740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:38.877385 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0b9e2-a554-4f9a-9036-b6c2412f2740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:38.877663 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0b9e2-a554-4f9a-9036-b6c2412f2740 admin admin] PATH_INFO: `/v3/projects/e1f470a894094b47926509bd7b9a8b65` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:38.878850 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a0b9e2-a554-4f9a-9036-b6c2412f2740 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1f470a894094b47926509bd7b9a8b65)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:38.880812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a0b9e2-a554-4f9a-9036-b6c2412f2740 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:38.947375 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-24a0b9e2-a554-4f9a-9036-b6c2412f2740 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1f470a894094b47926509bd7b9a8b65', 'request_id': 'req-24a0b9e2-a554-4f9a-9036-b6c2412f2740'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:38.953376 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-24a0b9e2-a554-4f9a-9036-b6c2412f2740 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1f470a894094b47926509bd7b9a8b65', 'request_id': 'req-24a0b9e2-a554-4f9a-9036-b6c2412f2740'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:38.960463 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-24a0b9e2-a554-4f9a-9036-b6c2412f2740 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1f470a894094b47926509bd7b9a8b65', 'request_id': 'req-24a0b9e2-a554-4f9a-9036-b6c2412f2740'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:38.972202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-24a0b9e2-a554-4f9a-9036-b6c2412f2740 admin admin] The token cache is being invalidate because project e1f470a894094b47926509bd7b9a8b65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:38.972472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-24a0b9e2-a554-4f9a-9036-b6c2412f2740 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:38.973097 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-24a0b9e2-a554-4f9a-9036-b6c2412f2740 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:38.974632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 930/1853] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:38 2022] DELETE /identity/v3/projects/e1f470a894094b47926509bd7b9a8b65 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:38:38.979442 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c859b88-ced2-4315-9355-eacde9763866 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:38.979951 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c859b88-ced2-4315-9355-eacde9763866 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:38.980062 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c859b88-ced2-4315-9355-eacde9763866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:38.998825 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3c859b88-ced2-4315-9355-eacde9763866 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:39.000045 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3c859b88-ced2-4315-9355-eacde9763866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:39.062414 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 924/1854] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:39.070400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f5c8e-7b7b-4a63-9052-3bc4f764f9f5 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:39.070922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f5c8e-7b7b-4a63-9052-3bc4f764f9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:39.071026 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f5c8e-7b7b-4a63-9052-3bc4f764f9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:39.088931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-d88f5c8e-7b7b-4a63-9052-3bc4f764f9f5 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:39.090121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-d88f5c8e-7b7b-4a63-9052-3bc4f764f9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:39.098620 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 931/1855] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:38:39.102853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c859b88-ced2-4315-9355-eacde9763866 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:39.113013 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c859b88-ced2-4315-9355-eacde9763866 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:39.117595 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e144b-e94f-4b95-b7ce-b9f24fbb91a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:39.118246 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e144b-e94f-4b95-b7ce-b9f24fbb91a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:39.118532 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e144b-e94f-4b95-b7ce-b9f24fbb91a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:39.118813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e144b-e94f-4b95-b7ce-b9f24fbb91a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:39.119533 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e144b-e94f-4b95-b7ce-b9f24fbb91a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:38:39.119880 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e144b-e94f-4b95-b7ce-b9f24fbb91a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:39.121468 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e144b-e94f-4b95-b7ce-b9f24fbb91a1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:39.133534 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 925/1856] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:38:39.137710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f5c8e-7b7b-4a63-9052-3bc4f764f9f5 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:39.147304 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f5c8e-7b7b-4a63-9052-3bc4f764f9f5 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:39.151942 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21810ffc-b5e3-49c4-8708-a141787b8e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:39.152581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21810ffc-b5e3-49c4-8708-a141787b8e74 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:39.152861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21810ffc-b5e3-49c4-8708-a141787b8e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:39.153135 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21810ffc-b5e3-49c4-8708-a141787b8e74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:39.153812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21810ffc-b5e3-49c4-8708-a141787b8e74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:39.155944 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21810ffc-b5e3-49c4-8708-a141787b8e74 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:39.180584 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 932/1857] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:39 2022] POST /identity/v3/projects => generated 381 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:39.184657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e144b-e94f-4b95-b7ce-b9f24fbb91a1 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:39.195333 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e144b-e94f-4b95-b7ce-b9f24fbb91a1 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:39.207341 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:39.251287 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2242c-1c44-4a5b-892f-1affdde1d847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:39.251978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2242c-1c44-4a5b-892f-1affdde1d847 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:39.252262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2242c-1c44-4a5b-892f-1affdde1d847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:39.252539 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2242c-1c44-4a5b-892f-1affdde1d847 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:39.253236 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b2242c-1c44-4a5b-892f-1affdde1d847 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:39.255136 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b2242c-1c44-4a5b-892f-1affdde1d847 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:39.274627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 926/1858] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:39 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:39.278767 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21810ffc-b5e3-49c4-8708-a141787b8e74 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:39.285885 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:39.289757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21810ffc-b5e3-49c4-8708-a141787b8e74 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:39.296706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab38a4b-d7b7-4b79-a988-59b4bade7533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:39.297385 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab38a4b-d7b7-4b79-a988-59b4bade7533 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:39.297849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab38a4b-d7b7-4b79-a988-59b4bade7533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:39.298276 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab38a4b-d7b7-4b79-a988-59b4bade7533 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:39.299000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab38a4b-d7b7-4b79-a988-59b4bade7533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:39.300623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab38a4b-d7b7-4b79-a988-59b4bade7533 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:39.311163 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 933/1859] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:39.315341 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2242c-1c44-4a5b-892f-1affdde1d847 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:39.325125 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2242c-1c44-4a5b-892f-1affdde1d847 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:39.329868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc3bcf-6376-4418-bb6d-167951563d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:39.330542 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc3bcf-6376-4418-bb6d-167951563d68 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:39.330823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc3bcf-6376-4418-bb6d-167951563d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:39.331095 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc3bcf-6376-4418-bb6d-167951563d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:39.331831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc3bcf-6376-4418-bb6d-167951563d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:39.333401 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc3bcf-6376-4418-bb6d-167951563d68 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:39.343793 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 927/1860] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:39.347716 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab38a4b-d7b7-4b79-a988-59b4bade7533 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:39.357621 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab38a4b-d7b7-4b79-a988-59b4bade7533 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:39.362534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380b33c-d98e-4cb9-a437-3100df9fe121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:39.363418 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380b33c-d98e-4cb9-a437-3100df9fe121 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:39.363698 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380b33c-d98e-4cb9-a437-3100df9fe121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:39.364010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380b33c-d98e-4cb9-a437-3100df9fe121 admin admin] PATH_INFO: `/v3/projects/9d569b202ad44d599c1771075a74be40/users/521738c69b6b446a8bf2a6cf320441d8/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:39.379317 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a380b33c-d98e-4cb9-a437-3100df9fe121 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d569b202ad44d599c1771075a74be40, user_id=521738c69b6b446a8bf2a6cf320441d8, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:39.382080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a380b33c-d98e-4cb9-a437-3100df9fe121 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:39.399410 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 934/1861] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:39 2022] PUT /identity/v3/projects/9d569b202ad44d599c1771075a74be40/users/521738c69b6b446a8bf2a6cf320441d8/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:39.409739 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e050df2-1b0b-4aa1-ba96-87f1a716226f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:39.410032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e050df2-1b0b-4aa1-ba96-87f1a716226f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:39.410310 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e050df2-1b0b-4aa1-ba96-87f1a716226f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:39.437240 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-7e050df2-1b0b-4aa1-ba96-87f1a716226f None None] MFA Rules not processed for user `521738c69b6b446a8bf2a6cf320441d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:39.438484 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-7e050df2-1b0b-4aa1-ba96-87f1a716226f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:39.501805 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 928/1862] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:39.506659 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380b33c-d98e-4cb9-a437-3100df9fe121 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:39.569532 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380b33c-d98e-4cb9-a437-3100df9fe121 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:39.574565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860701ef-3339-4ec0-80c4-377ab7bc0b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:39.575215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860701ef-3339-4ec0-80c4-377ab7bc0b1a admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:39.575493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860701ef-3339-4ec0-80c4-377ab7bc0b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:39.575765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860701ef-3339-4ec0-80c4-377ab7bc0b1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:39.577362 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860701ef-3339-4ec0-80c4-377ab7bc0b1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:39.579267 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860701ef-3339-4ec0-80c4-377ab7bc0b1a admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:39.604316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 935/1863] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:39 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:38:39.608127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e050df2-1b0b-4aa1-ba96-87f1a716226f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:39.618403 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e050df2-1b0b-4aa1-ba96-87f1a716226f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:39.675910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa9ce2-4168-4d1e-b1cb-828ed4c3c32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:39.676589 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa9ce2-4168-4d1e-b1cb-828ed4c3c32d admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:39.676874 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa9ce2-4168-4d1e-b1cb-828ed4c3c32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:39.677152 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa9ce2-4168-4d1e-b1cb-828ed4c3c32d admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:39.677851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa9ce2-4168-4d1e-b1cb-828ed4c3c32d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:39.679972 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa9ce2-4168-4d1e-b1cb-828ed4c3c32d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:39.704656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 929/1864] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:39 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:38:39.708509 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860701ef-3339-4ec0-80c4-377ab7bc0b1a admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:39.719223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860701ef-3339-4ec0-80c4-377ab7bc0b1a admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:39.724593 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888828c-5794-4065-a911-9a0f6f223057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:39.725259 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888828c-5794-4065-a911-9a0f6f223057 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:39.725543 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888828c-5794-4065-a911-9a0f6f223057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:39.725822 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888828c-5794-4065-a911-9a0f6f223057 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:39.726545 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888828c-5794-4065-a911-9a0f6f223057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:39.728166 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888828c-5794-4065-a911-9a0f6f223057 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:39.744657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 936/1865] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:39.748785 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa9ce2-4168-4d1e-b1cb-828ed4c3c32d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:39.759569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa9ce2-4168-4d1e-b1cb-828ed4c3c32d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:39.765309 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f62e9-1636-44b2-b85b-eb509f93b547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:39.766279 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f62e9-1636-44b2-b85b-eb509f93b547 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:39.766629 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f62e9-1636-44b2-b85b-eb509f93b547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:39.766975 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f62e9-1636-44b2-b85b-eb509f93b547 admin admin] PATH_INFO: `/v3/projects/5f053444e83a43b4aafcbf0ac069c6f5/users/e04b54c67ee649a1abf61e0e89a0bd46/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:39.780851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:39.784271 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f62e9-1636-44b2-b85b-eb509f93b547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f053444e83a43b4aafcbf0ac069c6f5, user_id=e04b54c67ee649a1abf61e0e89a0bd46, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:39.787539 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f62e9-1636-44b2-b85b-eb509f93b547 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:39.803156 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 930/1866] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:39 2022] PUT /identity/v3/projects/5f053444e83a43b4aafcbf0ac069c6f5/users/e04b54c67ee649a1abf61e0e89a0bd46/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:39.807153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888828c-5794-4065-a911-9a0f6f223057 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:39.819170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888828c-5794-4065-a911-9a0f6f223057 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:39.878715 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63396eb-2722-4f05-b7d2-ce672a429323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:39.879485 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63396eb-2722-4f05-b7d2-ce672a429323 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:39.879874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63396eb-2722-4f05-b7d2-ce672a429323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:39.880218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63396eb-2722-4f05-b7d2-ce672a429323 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:39.881016 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63396eb-2722-4f05-b7d2-ce672a429323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:39.882606 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63396eb-2722-4f05-b7d2-ce672a429323 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:39.896559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 937/1867] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:39.897737 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f62e9-1636-44b2-b85b-eb509f93b547 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:39.908290 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f62e9-1636-44b2-b85b-eb509f93b547 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:39.913980 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340122cd-22a2-44e8-a10e-69dbe9ec19de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:39.914881 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340122cd-22a2-44e8-a10e-69dbe9ec19de admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:39.915169 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340122cd-22a2-44e8-a10e-69dbe9ec19de admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:39.915445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340122cd-22a2-44e8-a10e-69dbe9ec19de admin admin] PATH_INFO: `/v3/projects/5f053444e83a43b4aafcbf0ac069c6f5/users/e04b54c67ee649a1abf61e0e89a0bd46/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:39.917505 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340122cd-22a2-44e8-a10e-69dbe9ec19de admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f053444e83a43b4aafcbf0ac069c6f5, user_id=e04b54c67ee649a1abf61e0e89a0bd46, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:39.927027 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340122cd-22a2-44e8-a10e-69dbe9ec19de admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:39.936104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 931/1868] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:39 2022] PUT /identity/v3/projects/5f053444e83a43b4aafcbf0ac069c6f5/users/e04b54c67ee649a1abf61e0e89a0bd46/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:39.950575 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076aa10b-a0c3-4938-adf7-43ddbb94c2d4 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:39.950575 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076aa10b-a0c3-4938-adf7-43ddbb94c2d4 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:39.950575 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076aa10b-a0c3-4938-adf7-43ddbb94c2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:39.980615 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-076aa10b-a0c3-4938-adf7-43ddbb94c2d4 None None] MFA Rules not processed for user `e04b54c67ee649a1abf61e0e89a0bd46`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:39.981998 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-076aa10b-a0c3-4938-adf7-43ddbb94c2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:40.014604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:40.062679 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 938/1869] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:40.075138 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340122cd-22a2-44e8-a10e-69dbe9ec19de admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:40.077020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-340122cd-22a2-44e8-a10e-69dbe9ec19de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:40.162825 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340122cd-22a2-44e8-a10e-69dbe9ec19de admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:40.167755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef71992-b8e3-4127-86ee-e069347d0721 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:40.168544 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef71992-b8e3-4127-86ee-e069347d0721 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:40.168901 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef71992-b8e3-4127-86ee-e069347d0721 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:40.169250 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef71992-b8e3-4127-86ee-e069347d0721 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:40.180499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef71992-b8e3-4127-86ee-e069347d0721 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:40.182122 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef71992-b8e3-4127-86ee-e069347d0721 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:40.192146 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 932/1870] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:40.289086 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076aa10b-a0c3-4938-adf7-43ddbb94c2d4 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:40.301095 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076aa10b-a0c3-4938-adf7-43ddbb94c2d4 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:40.305882 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986aed6-38d7-443c-9001-7a3278698224 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:40.306886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986aed6-38d7-443c-9001-7a3278698224 service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:40.307341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986aed6-38d7-443c-9001-7a3278698224 service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:40.307707 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986aed6-38d7-443c-9001-7a3278698224 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:40.372766 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2986aed6-38d7-443c-9001-7a3278698224 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:40.374500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2986aed6-38d7-443c-9001-7a3278698224 service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:40.385497 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 939/1871] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:40.675146 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef71992-b8e3-4127-86ee-e069347d0721 service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:40.686888 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef71992-b8e3-4127-86ee-e069347d0721 service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:40.691556 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85c854-991f-4237-a95a-b169d99d6040 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:40.691818 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85c854-991f-4237-a95a-b169d99d6040 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:40.692082 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85c854-991f-4237-a95a-b169d99d6040 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:40.692365 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85c854-991f-4237-a95a-b169d99d6040 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:40.695547 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1a85c854-991f-4237-a95a-b169d99d6040 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:40.779042 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85c854-991f-4237-a95a-b169d99d6040 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:40.781398 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85c854-991f-4237-a95a-b169d99d6040 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:40.792951 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 933/1872] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:40 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:41.093365 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986aed6-38d7-443c-9001-7a3278698224 service cinder] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:41.200568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986aed6-38d7-443c-9001-7a3278698224 service cinder] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:41.205597 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b4036-6fae-4469-9a55-084475e199c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:41.206642 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b4036-6fae-4469-9a55-084475e199c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:41.206933 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b4036-6fae-4469-9a55-084475e199c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:41.207863 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b4036-6fae-4469-9a55-084475e199c8 admin admin] PATH_INFO: `/v3/users/521738c69b6b446a8bf2a6cf320441d8` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:41.209650 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b4036-6fae-4469-9a55-084475e199c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=521738c69b6b446a8bf2a6cf320441d8)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:41.212003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b4036-6fae-4469-9a55-084475e199c8 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:41.274238 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-c65b4036-6fae-4469-9a55-084475e199c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '521738c69b6b446a8bf2a6cf320441d8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:41.284405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-c65b4036-6fae-4469-9a55-084475e199c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '521738c69b6b446a8bf2a6cf320441d8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:41.318117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-c65b4036-6fae-4469-9a55-084475e199c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '521738c69b6b446a8bf2a6cf320441d8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:41.319175 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-c65b4036-6fae-4469-9a55-084475e199c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '521738c69b6b446a8bf2a6cf320441d8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:41.331191 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:41.344537 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-c65b4036-6fae-4469-9a55-084475e199c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '521738c69b6b446a8bf2a6cf320441d8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:41.354401 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 940/1873] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:41 2022] DELETE /identity/v3/users/521738c69b6b446a8bf2a6cf320441d8 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:41.361034 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34792df8-59bf-4fe2-806d-16d8f05adf3d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:41.361552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34792df8-59bf-4fe2-806d-16d8f05adf3d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:41.361906 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34792df8-59bf-4fe2-806d-16d8f05adf3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:41.409923 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-34792df8-59bf-4fe2-806d-16d8f05adf3d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:41.412461 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-34792df8-59bf-4fe2-806d-16d8f05adf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:41.587825 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 934/1874] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:41.596207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b4036-6fae-4469-9a55-084475e199c8 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:41.598226 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-c65b4036-6fae-4469-9a55-084475e199c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:41.692327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b4036-6fae-4469-9a55-084475e199c8 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:41.700333 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ac8f4-7b83-4d79-9ab0-5105c91025a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:41.701315 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ac8f4-7b83-4d79-9ab0-5105c91025a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:41.701690 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ac8f4-7b83-4d79-9ab0-5105c91025a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:41.702059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ac8f4-7b83-4d79-9ab0-5105c91025a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:41.719892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ac8f4-7b83-4d79-9ab0-5105c91025a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:41.721976 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ac8f4-7b83-4d79-9ab0-5105c91025a7 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:41.758532 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 941/1875] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:42.545587 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34792df8-59bf-4fe2-806d-16d8f05adf3d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:42.547244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-34792df8-59bf-4fe2-806d-16d8f05adf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:42.561483 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34792df8-59bf-4fe2-806d-16d8f05adf3d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:42.569939 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd3b02-eccc-4407-9543-277d15a3c246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:42.571201 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd3b02-eccc-4407-9543-277d15a3c246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:42.571488 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd3b02-eccc-4407-9543-277d15a3c246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:42.571958 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd3b02-eccc-4407-9543-277d15a3c246 admin admin] PATH_INFO: `/v3/projects/9d569b202ad44d599c1771075a74be40` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:42.573211 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd3b02-eccc-4407-9543-277d15a3c246 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d569b202ad44d599c1771075a74be40)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:42.575237 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd3b02-eccc-4407-9543-277d15a3c246 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:42.589774 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:42.671617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-69fd3b02-eccc-4407-9543-277d15a3c246 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d569b202ad44d599c1771075a74be40', 'request_id': 'req-69fd3b02-eccc-4407-9543-277d15a3c246'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:42.683403 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-69fd3b02-eccc-4407-9543-277d15a3c246 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d569b202ad44d599c1771075a74be40', 'request_id': 'req-69fd3b02-eccc-4407-9543-277d15a3c246'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:42.696358 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-69fd3b02-eccc-4407-9543-277d15a3c246 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d569b202ad44d599c1771075a74be40', 'request_id': 'req-69fd3b02-eccc-4407-9543-277d15a3c246'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:42.721728 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-69fd3b02-eccc-4407-9543-277d15a3c246 admin admin] The token cache is being invalidate because project 9d569b202ad44d599c1771075a74be40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:42.722175 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-69fd3b02-eccc-4407-9543-277d15a3c246 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:42.723269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-69fd3b02-eccc-4407-9543-277d15a3c246 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:42.729160 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 935/1876] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:42 2022] DELETE /identity/v3/projects/9d569b202ad44d599c1771075a74be40 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:42.732777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ac8f4-7b83-4d79-9ab0-5105c91025a7 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:42.752110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ac8f4-7b83-4d79-9ab0-5105c91025a7 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:42.881101 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b54054-c18c-4c79-82d9-f9e20f458847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:42.881622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b54054-c18c-4c79-82d9-f9e20f458847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:42.881965 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b54054-c18c-4c79-82d9-f9e20f458847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:42.882188 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b54054-c18c-4c79-82d9-f9e20f458847 admin admin] PATH_INFO: `/v3/users/e04b54c67ee649a1abf61e0e89a0bd46` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:42.883533 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b54054-c18c-4c79-82d9-f9e20f458847 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e04b54c67ee649a1abf61e0e89a0bd46)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:42.885509 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b54054-c18c-4c79-82d9-f9e20f458847 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:42.977772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-90b54054-c18c-4c79-82d9-f9e20f458847 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e04b54c67ee649a1abf61e0e89a0bd46'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:42.989724 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-90b54054-c18c-4c79-82d9-f9e20f458847 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e04b54c67ee649a1abf61e0e89a0bd46'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:43.008736 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-90b54054-c18c-4c79-82d9-f9e20f458847 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e04b54c67ee649a1abf61e0e89a0bd46'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:43.009841 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-90b54054-c18c-4c79-82d9-f9e20f458847 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e04b54c67ee649a1abf61e0e89a0bd46'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:43.035185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd3b02-eccc-4407-9543-277d15a3c246 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:43.035415 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-90b54054-c18c-4c79-82d9-f9e20f458847 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e04b54c67ee649a1abf61e0e89a0bd46'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:43.037342 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-69fd3b02-eccc-4407-9543-277d15a3c246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:43.057335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 942/1877] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:42 2022] DELETE /identity/v3/users/e04b54c67ee649a1abf61e0e89a0bd46 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:43.162168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd3b02-eccc-4407-9543-277d15a3c246 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:43.240827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc900f5-2209-4f7a-bb85-d777d5454a38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:43.241491 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc900f5-2209-4f7a-bb85-d777d5454a38 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:43.241766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc900f5-2209-4f7a-bb85-d777d5454a38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:43.242037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc900f5-2209-4f7a-bb85-d777d5454a38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:43.244698 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ffc900f5-2209-4f7a-bb85-d777d5454a38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:43.401827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc900f5-2209-4f7a-bb85-d777d5454a38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:43.404695 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc900f5-2209-4f7a-bb85-d777d5454a38 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:43.425067 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 936/1878] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:38:44.678229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b54054-c18c-4c79-82d9-f9e20f458847 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:44.681574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-90b54054-c18c-4c79-82d9-f9e20f458847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:44.692121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:44.764948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b54054-c18c-4c79-82d9-f9e20f458847 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:44.771740 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a93fe-056f-4417-b31d-c000797367e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:44.772764 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a93fe-056f-4417-b31d-c000797367e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:44.773123 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a93fe-056f-4417-b31d-c000797367e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:44.773464 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a93fe-056f-4417-b31d-c000797367e7 admin admin] PATH_INFO: `/v3/projects/5f053444e83a43b4aafcbf0ac069c6f5` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:44.775081 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a93fe-056f-4417-b31d-c000797367e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f053444e83a43b4aafcbf0ac069c6f5)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:44.777300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a93fe-056f-4417-b31d-c000797367e7 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:44.846024 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-cd0a93fe-056f-4417-b31d-c000797367e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f053444e83a43b4aafcbf0ac069c6f5', 'request_id': 'req-cd0a93fe-056f-4417-b31d-c000797367e7'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:44.853847 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-cd0a93fe-056f-4417-b31d-c000797367e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f053444e83a43b4aafcbf0ac069c6f5', 'request_id': 'req-cd0a93fe-056f-4417-b31d-c000797367e7'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:44.868517 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-cd0a93fe-056f-4417-b31d-c000797367e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f053444e83a43b4aafcbf0ac069c6f5', 'request_id': 'req-cd0a93fe-056f-4417-b31d-c000797367e7'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:44.891883 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-cd0a93fe-056f-4417-b31d-c000797367e7 admin admin] The token cache is being invalidate because project 5f053444e83a43b4aafcbf0ac069c6f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:44.892490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-cd0a93fe-056f-4417-b31d-c000797367e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:44.893611 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-cd0a93fe-056f-4417-b31d-c000797367e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:44.896192 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 943/1879] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:44 2022] DELETE /identity/v3/projects/5f053444e83a43b4aafcbf0ac069c6f5 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:44.901840 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d55aeb-abbc-4ef4-b5a0-8fa4bb453003 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:44.902742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d55aeb-abbc-4ef4-b5a0-8fa4bb453003 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:44.903299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d55aeb-abbc-4ef4-b5a0-8fa4bb453003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:44.925787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-61d55aeb-abbc-4ef4-b5a0-8fa4bb453003 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:44.927249 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-61d55aeb-abbc-4ef4-b5a0-8fa4bb453003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:44.933754 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:45.052851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 937/1880] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:38:45.060227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e61a8-19df-4d8d-82db-2aa2724a6633 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:45.060520 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e61a8-19df-4d8d-82db-2aa2724a6633 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:45.060803 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e61a8-19df-4d8d-82db-2aa2724a6633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:45.090932 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-c48e61a8-19df-4d8d-82db-2aa2724a6633 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:45.093484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-c48e61a8-19df-4d8d-82db-2aa2724a6633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:45.107521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 944/1881] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:45.117118 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d55aeb-abbc-4ef4-b5a0-8fa4bb453003 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:45.130171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d55aeb-abbc-4ef4-b5a0-8fa4bb453003 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:45.137342 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfb67d-56ae-493b-b1be-5c60903a6c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:45.138069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfb67d-56ae-493b-b1be-5c60903a6c54 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:45.139111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfb67d-56ae-493b-b1be-5c60903a6c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:45.139397 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfb67d-56ae-493b-b1be-5c60903a6c54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:45.140831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfb67d-56ae-493b-b1be-5c60903a6c54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:38:45.141198 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfb67d-56ae-493b-b1be-5c60903a6c54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:45.143286 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfb67d-56ae-493b-b1be-5c60903a6c54 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:45.166437 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 938/1882] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:38:45.168516 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e61a8-19df-4d8d-82db-2aa2724a6633 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:45.184529 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e61a8-19df-4d8d-82db-2aa2724a6633 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:45.209139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094df83-43c0-48ed-ae34-01d2a64d3595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:45.210772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094df83-43c0-48ed-ae34-01d2a64d3595 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:45.211062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094df83-43c0-48ed-ae34-01d2a64d3595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:45.211411 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094df83-43c0-48ed-ae34-01d2a64d3595 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:45.212219 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094df83-43c0-48ed-ae34-01d2a64d3595 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:45.214476 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094df83-43c0-48ed-ae34-01d2a64d3595 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:45.265081 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 945/1883] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:45 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:45.266227 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfb67d-56ae-493b-b1be-5c60903a6c54 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:45.279823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfb67d-56ae-493b-b1be-5c60903a6c54 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:45.390925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7259ea5-ea45-4c5c-a6cb-ca15687ff353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:45.391839 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7259ea5-ea45-4c5c-a6cb-ca15687ff353 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:45.392203 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7259ea5-ea45-4c5c-a6cb-ca15687ff353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:45.392550 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7259ea5-ea45-4c5c-a6cb-ca15687ff353 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:45.393357 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7259ea5-ea45-4c5c-a6cb-ca15687ff353 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:45.395749 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7259ea5-ea45-4c5c-a6cb-ca15687ff353 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:45.440143 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 939/1884] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:45 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:45.446098 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094df83-43c0-48ed-ae34-01d2a64d3595 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:45.464190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094df83-43c0-48ed-ae34-01d2a64d3595 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:45.471387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838976f3-4ab5-48bd-a4ef-c77314f51fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:45.473029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838976f3-4ab5-48bd-a4ef-c77314f51fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:45.473338 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838976f3-4ab5-48bd-a4ef-c77314f51fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:45.473622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838976f3-4ab5-48bd-a4ef-c77314f51fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:45.474723 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838976f3-4ab5-48bd-a4ef-c77314f51fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:45.476503 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838976f3-4ab5-48bd-a4ef-c77314f51fb7 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:45.496882 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 946/1885] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:45.498783 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7259ea5-ea45-4c5c-a6cb-ca15687ff353 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:45.522992 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7259ea5-ea45-4c5c-a6cb-ca15687ff353 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:45.541415 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1599ce-a7d7-41cf-b2c6-3499df9b7928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:45.544090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1599ce-a7d7-41cf-b2c6-3499df9b7928 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:45.546208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1599ce-a7d7-41cf-b2c6-3499df9b7928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:45.547375 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1599ce-a7d7-41cf-b2c6-3499df9b7928 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:45.550611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1599ce-a7d7-41cf-b2c6-3499df9b7928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:45.557639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1599ce-a7d7-41cf-b2c6-3499df9b7928 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:45.586225 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 940/1886] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:38:45.594039 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838976f3-4ab5-48bd-a4ef-c77314f51fb7 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:45.616120 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838976f3-4ab5-48bd-a4ef-c77314f51fb7 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:45.624278 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248dfa1-6841-415e-92a2-d78454f2e8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:45.625425 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248dfa1-6841-415e-92a2-d78454f2e8ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:45.625825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248dfa1-6841-415e-92a2-d78454f2e8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:45.626307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248dfa1-6841-415e-92a2-d78454f2e8ad admin admin] PATH_INFO: `/v3/projects/2dce49332f3d4ce6a2976f653418add4/users/44e52bd61f8241dabb1b81fb5ece491f/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:45.659741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248dfa1-6841-415e-92a2-d78454f2e8ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dce49332f3d4ce6a2976f653418add4, user_id=44e52bd61f8241dabb1b81fb5ece491f, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:45.662903 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248dfa1-6841-415e-92a2-d78454f2e8ad admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:45.694136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 947/1887] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:45 2022] PUT /identity/v3/projects/2dce49332f3d4ce6a2976f653418add4/users/44e52bd61f8241dabb1b81fb5ece491f/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:45.695568 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34a21a-523f-43a5-b37d-a389af3626ee None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:45.695755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34a21a-523f-43a5-b37d-a389af3626ee None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:45.696100 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34a21a-523f-43a5-b37d-a389af3626ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:45.730177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-0c34a21a-523f-43a5-b37d-a389af3626ee None None] MFA Rules not processed for user `44e52bd61f8241dabb1b81fb5ece491f`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:45.731546 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0c34a21a-523f-43a5-b37d-a389af3626ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:45.804368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 941/1888] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:45.815534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248dfa1-6841-415e-92a2-d78454f2e8ad admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:45.817333 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-b248dfa1-6841-415e-92a2-d78454f2e8ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:45.854594 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:45.883930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248dfa1-6841-415e-92a2-d78454f2e8ad admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:45.887660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87769e9-8fc8-4722-ad8d-58cbee275611 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:45.888527 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87769e9-8fc8-4722-ad8d-58cbee275611 service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:45.888865 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87769e9-8fc8-4722-ad8d-58cbee275611 service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:45.889141 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87769e9-8fc8-4722-ad8d-58cbee275611 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:45.898114 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87769e9-8fc8-4722-ad8d-58cbee275611 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:45.899566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87769e9-8fc8-4722-ad8d-58cbee275611 service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:45.908870 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 948/1889] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:45.964743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34a21a-523f-43a5-b37d-a389af3626ee None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:45.966292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0c34a21a-523f-43a5-b37d-a389af3626ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:45.972818 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:46.039121 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34a21a-523f-43a5-b37d-a389af3626ee None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:46.049245 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e42eaa-ff46-4a4f-a35f-4c55e4741578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:46.050357 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e42eaa-ff46-4a4f-a35f-4c55e4741578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:46.050804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e42eaa-ff46-4a4f-a35f-4c55e4741578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:46.051224 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e42eaa-ff46-4a4f-a35f-4c55e4741578 admin admin] PATH_INFO: `/v3/users/78b6d6dd87bc41b191ab96ad6fc9068b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:46.053304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e42eaa-ff46-4a4f-a35f-4c55e4741578 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b6d6dd87bc41b191ab96ad6fc9068b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:46.055463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e42eaa-ff46-4a4f-a35f-4c55e4741578 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:46.164884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f8e42eaa-ff46-4a4f-a35f-4c55e4741578 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b6d6dd87bc41b191ab96ad6fc9068b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:46.176649 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f8e42eaa-ff46-4a4f-a35f-4c55e4741578 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b6d6dd87bc41b191ab96ad6fc9068b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:46.177929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f8e42eaa-ff46-4a4f-a35f-4c55e4741578 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b6d6dd87bc41b191ab96ad6fc9068b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:46.179065 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f8e42eaa-ff46-4a4f-a35f-4c55e4741578 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b6d6dd87bc41b191ab96ad6fc9068b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:46.202119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f8e42eaa-ff46-4a4f-a35f-4c55e4741578 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b6d6dd87bc41b191ab96ad6fc9068b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:46.225153 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 942/1890] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:45 2022] DELETE /identity/v3/users/78b6d6dd87bc41b191ab96ad6fc9068b => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:46.229581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e640172-3769-4e81-80ce-33ad0e1ce8ff None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:46.229956 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e640172-3769-4e81-80ce-33ad0e1ce8ff None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:46.230329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e640172-3769-4e81-80ce-33ad0e1ce8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:46.273811 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-6e640172-3769-4e81-80ce-33ad0e1ce8ff None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:46.276557 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-6e640172-3769-4e81-80ce-33ad0e1ce8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:46.365377 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 949/1891] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:38:46.375369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e42eaa-ff46-4a4f-a35f-4c55e4741578 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:46.376866 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f8e42eaa-ff46-4a4f-a35f-4c55e4741578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:46.459823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e42eaa-ff46-4a4f-a35f-4c55e4741578 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:46.465444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ca217-765b-46a5-8727-b5f623a644ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:46.466379 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ca217-765b-46a5-8727-b5f623a644ac service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:46.467102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ca217-765b-46a5-8727-b5f623a644ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:46.467369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ca217-765b-46a5-8727-b5f623a644ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:46.484914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ca217-765b-46a5-8727-b5f623a644ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:46.487712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ca217-765b-46a5-8727-b5f623a644ac service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:46.504266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 943/1892] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:46.974192 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e640172-3769-4e81-80ce-33ad0e1ce8ff None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:46.976226 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-6e640172-3769-4e81-80ce-33ad0e1ce8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:47.000235 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e640172-3769-4e81-80ce-33ad0e1ce8ff None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:47.005854 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cacf80-1721-441d-9f6a-5f71ea0843d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:47.006733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cacf80-1721-441d-9f6a-5f71ea0843d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:47.007021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cacf80-1721-441d-9f6a-5f71ea0843d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:47.007303 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cacf80-1721-441d-9f6a-5f71ea0843d2 admin admin] PATH_INFO: `/v3/projects/50b8259163124d649ffa6202dab55736` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:47.008512 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cacf80-1721-441d-9f6a-5f71ea0843d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50b8259163124d649ffa6202dab55736)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:47.010996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cacf80-1721-441d-9f6a-5f71ea0843d2 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:47.075130 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-45cacf80-1721-441d-9f6a-5f71ea0843d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50b8259163124d649ffa6202dab55736', 'request_id': 'req-45cacf80-1721-441d-9f6a-5f71ea0843d2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:47.087465 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-45cacf80-1721-441d-9f6a-5f71ea0843d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50b8259163124d649ffa6202dab55736', 'request_id': 'req-45cacf80-1721-441d-9f6a-5f71ea0843d2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:47.098589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-45cacf80-1721-441d-9f6a-5f71ea0843d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50b8259163124d649ffa6202dab55736', 'request_id': 'req-45cacf80-1721-441d-9f6a-5f71ea0843d2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:47.116535 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-45cacf80-1721-441d-9f6a-5f71ea0843d2 admin admin] The token cache is being invalidate because project 50b8259163124d649ffa6202dab55736 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:47.116879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-45cacf80-1721-441d-9f6a-5f71ea0843d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:47.118049 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-45cacf80-1721-441d-9f6a-5f71ea0843d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:47.120081 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 950/1893] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:46 2022] DELETE /identity/v3/projects/50b8259163124d649ffa6202dab55736 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:47.124800 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ca217-765b-46a5-8727-b5f623a644ac service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:47.139700 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ca217-765b-46a5-8727-b5f623a644ac service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:47.180549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:47.242586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9c1b9-c84e-47ec-a66a-992ab202f969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:47.243482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9c1b9-c84e-47ec-a66a-992ab202f969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:47.243765 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9c1b9-c84e-47ec-a66a-992ab202f969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:47.244068 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9c1b9-c84e-47ec-a66a-992ab202f969 admin admin] PATH_INFO: `/v3/users/57966351a15d443eb7fe9c46b3bea34d` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:47.245302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9c1b9-c84e-47ec-a66a-992ab202f969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57966351a15d443eb7fe9c46b3bea34d)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:47.247567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9c1b9-c84e-47ec-a66a-992ab202f969 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:47.298255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f0d9c1b9-c84e-47ec-a66a-992ab202f969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57966351a15d443eb7fe9c46b3bea34d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:47.306319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f0d9c1b9-c84e-47ec-a66a-992ab202f969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57966351a15d443eb7fe9c46b3bea34d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:47.307010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f0d9c1b9-c84e-47ec-a66a-992ab202f969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57966351a15d443eb7fe9c46b3bea34d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:47.307607 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f0d9c1b9-c84e-47ec-a66a-992ab202f969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57966351a15d443eb7fe9c46b3bea34d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:47.320892 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f0d9c1b9-c84e-47ec-a66a-992ab202f969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57966351a15d443eb7fe9c46b3bea34d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:47.337208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 944/1894] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:47 2022] DELETE /identity/v3/users/57966351a15d443eb7fe9c46b3bea34d => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:47.832464 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cacf80-1721-441d-9f6a-5f71ea0843d2 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:47.834077 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-45cacf80-1721-441d-9f6a-5f71ea0843d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:47.884806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cacf80-1721-441d-9f6a-5f71ea0843d2 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:48.104853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:48.118268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f213331-d458-41cd-ac13-40d020e6da70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:48.119411 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f213331-d458-41cd-ac13-40d020e6da70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:48.119875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f213331-d458-41cd-ac13-40d020e6da70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:48.120298 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f213331-d458-41cd-ac13-40d020e6da70 admin admin] PATH_INFO: `/v3/projects/1e8e29f8eb004f96a5cd17a03cc27c19` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:48.121922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f213331-d458-41cd-ac13-40d020e6da70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e8e29f8eb004f96a5cd17a03cc27c19)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:48.127597 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f213331-d458-41cd-ac13-40d020e6da70 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:48.221705 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2f213331-d458-41cd-ac13-40d020e6da70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e8e29f8eb004f96a5cd17a03cc27c19', 'request_id': 'req-2f213331-d458-41cd-ac13-40d020e6da70'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:48.228340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2f213331-d458-41cd-ac13-40d020e6da70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e8e29f8eb004f96a5cd17a03cc27c19', 'request_id': 'req-2f213331-d458-41cd-ac13-40d020e6da70'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:48.243246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2f213331-d458-41cd-ac13-40d020e6da70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e8e29f8eb004f96a5cd17a03cc27c19', 'request_id': 'req-2f213331-d458-41cd-ac13-40d020e6da70'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:48.308819 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2f213331-d458-41cd-ac13-40d020e6da70 admin admin] The token cache is being invalidate because project 1e8e29f8eb004f96a5cd17a03cc27c19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:48.309138 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2f213331-d458-41cd-ac13-40d020e6da70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:48.315135 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2f213331-d458-41cd-ac13-40d020e6da70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:48.318955 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 951/1895] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:47 2022] DELETE /identity/v3/projects/1e8e29f8eb004f96a5cd17a03cc27c19 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:48.323819 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9c1b9-c84e-47ec-a66a-992ab202f969 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:48.326854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f0d9c1b9-c84e-47ec-a66a-992ab202f969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:48.351948 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9c1b9-c84e-47ec-a66a-992ab202f969 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:48.429834 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67db1a8-1164-42b4-8889-d7b3b814e807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:48.431027 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67db1a8-1164-42b4-8889-d7b3b814e807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:48.431605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67db1a8-1164-42b4-8889-d7b3b814e807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:48.432117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67db1a8-1164-42b4-8889-d7b3b814e807 admin admin] PATH_INFO: `/v3/users/7217444e28154916acb4a802eb4e9704` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:48.434364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67db1a8-1164-42b4-8889-d7b3b814e807 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7217444e28154916acb4a802eb4e9704)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:48.438855 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67db1a8-1164-42b4-8889-d7b3b814e807 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:48.502685 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e67db1a8-1164-42b4-8889-d7b3b814e807 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7217444e28154916acb4a802eb4e9704'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:48.504418 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:48.521150 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e67db1a8-1164-42b4-8889-d7b3b814e807 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7217444e28154916acb4a802eb4e9704'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:48.522084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e67db1a8-1164-42b4-8889-d7b3b814e807 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7217444e28154916acb4a802eb4e9704'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:48.522795 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e67db1a8-1164-42b4-8889-d7b3b814e807 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7217444e28154916acb4a802eb4e9704'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:48.560946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e67db1a8-1164-42b4-8889-d7b3b814e807 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7217444e28154916acb4a802eb4e9704'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:48.605333 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 945/1896] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:48 2022] DELETE /identity/v3/users/7217444e28154916acb4a802eb4e9704 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:49.049793 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f213331-d458-41cd-ac13-40d020e6da70 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:49.051490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-2f213331-d458-41cd-ac13-40d020e6da70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:49.065981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f213331-d458-41cd-ac13-40d020e6da70 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:49.124177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa18cd-518c-4733-9690-3e83f19db5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:49.124980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa18cd-518c-4733-9690-3e83f19db5d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:49.125260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa18cd-518c-4733-9690-3e83f19db5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:49.125532 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa18cd-518c-4733-9690-3e83f19db5d6 admin admin] PATH_INFO: `/v3/projects/c82db829db6144e9a93adaa8010e0854` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:49.126831 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fa18cd-518c-4733-9690-3e83f19db5d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c82db829db6144e9a93adaa8010e0854)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:49.128763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fa18cd-518c-4733-9690-3e83f19db5d6 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:49.183606 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b4fa18cd-518c-4733-9690-3e83f19db5d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c82db829db6144e9a93adaa8010e0854', 'request_id': 'req-b4fa18cd-518c-4733-9690-3e83f19db5d6'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:49.192847 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b4fa18cd-518c-4733-9690-3e83f19db5d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c82db829db6144e9a93adaa8010e0854', 'request_id': 'req-b4fa18cd-518c-4733-9690-3e83f19db5d6'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:49.206291 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b4fa18cd-518c-4733-9690-3e83f19db5d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c82db829db6144e9a93adaa8010e0854', 'request_id': 'req-b4fa18cd-518c-4733-9690-3e83f19db5d6'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:49.230612 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b4fa18cd-518c-4733-9690-3e83f19db5d6 admin admin] The token cache is being invalidate because project c82db829db6144e9a93adaa8010e0854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:49.231298 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b4fa18cd-518c-4733-9690-3e83f19db5d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:49.232813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b4fa18cd-518c-4733-9690-3e83f19db5d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:49.234657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 952/1897] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:49 2022] DELETE /identity/v3/projects/c82db829db6144e9a93adaa8010e0854 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:49.241421 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec7d72-4738-4ecf-aa5b-bd5e2f2f56fa None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:49.241721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec7d72-4738-4ecf-aa5b-bd5e2f2f56fa None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:49.242005 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec7d72-4738-4ecf-aa5b-bd5e2f2f56fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:49.262557 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ecec7d72-4738-4ecf-aa5b-bd5e2f2f56fa None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:49.264061 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ecec7d72-4738-4ecf-aa5b-bd5e2f2f56fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:49.350567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 946/1898] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:38:49.359827 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475929a6-a547-4104-9a0c-e32be8794247 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:49.360172 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475929a6-a547-4104-9a0c-e32be8794247 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:49.360482 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475929a6-a547-4104-9a0c-e32be8794247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:49.406593 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-475929a6-a547-4104-9a0c-e32be8794247 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:49.407975 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-475929a6-a547-4104-9a0c-e32be8794247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:49.417971 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 953/1899] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:49.421914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec7d72-4738-4ecf-aa5b-bd5e2f2f56fa None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:49.432508 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec7d72-4738-4ecf-aa5b-bd5e2f2f56fa None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:49.438565 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db20e45-1639-4f06-9e7d-5871c3c52c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:49.439483 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db20e45-1639-4f06-9e7d-5871c3c52c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:49.440107 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db20e45-1639-4f06-9e7d-5871c3c52c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:49.440578 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db20e45-1639-4f06-9e7d-5871c3c52c2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:49.441493 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db20e45-1639-4f06-9e7d-5871c3c52c2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:38:49.441967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db20e45-1639-4f06-9e7d-5871c3c52c2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:49.443964 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db20e45-1639-4f06-9e7d-5871c3c52c2a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:49.461430 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 947/1900] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:38:49.463006 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475929a6-a547-4104-9a0c-e32be8794247 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:49.474803 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475929a6-a547-4104-9a0c-e32be8794247 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:49.487877 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ee62a-0397-411a-9c79-0db2456b7dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:49.488567 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ee62a-0397-411a-9c79-0db2456b7dcd admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:49.488981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ee62a-0397-411a-9c79-0db2456b7dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:49.489890 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ee62a-0397-411a-9c79-0db2456b7dcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:49.490981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ee62a-0397-411a-9c79-0db2456b7dcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:49.493644 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ee62a-0397-411a-9c79-0db2456b7dcd admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:49.509085 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:49.542859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 954/1901] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:49 2022] POST /identity/v3/projects => generated 383 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:38:49.544010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db20e45-1639-4f06-9e7d-5871c3c52c2a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:49.554462 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db20e45-1639-4f06-9e7d-5871c3c52c2a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:49.612763 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096aaac8-7534-4da3-a825-509bb5eba8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:49.613493 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096aaac8-7534-4da3-a825-509bb5eba8f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:49.613829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096aaac8-7534-4da3-a825-509bb5eba8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:49.614191 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096aaac8-7534-4da3-a825-509bb5eba8f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:49.614885 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096aaac8-7534-4da3-a825-509bb5eba8f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:49.616836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096aaac8-7534-4da3-a825-509bb5eba8f2 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:49.637066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 948/1902] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:49 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:49.642205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ee62a-0397-411a-9c79-0db2456b7dcd admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:49.657136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ee62a-0397-411a-9c79-0db2456b7dcd admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:49.662345 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5188da1f-08d8-4ab3-bf7e-c91d84098818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:49.663199 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188da1f-08d8-4ab3-bf7e-c91d84098818 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:49.663620 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188da1f-08d8-4ab3-bf7e-c91d84098818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:49.664017 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188da1f-08d8-4ab3-bf7e-c91d84098818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:49.664867 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5188da1f-08d8-4ab3-bf7e-c91d84098818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:49.666487 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5188da1f-08d8-4ab3-bf7e-c91d84098818 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:49.680102 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 955/1903] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:38:49.684528 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096aaac8-7534-4da3-a825-509bb5eba8f2 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:49.707420 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096aaac8-7534-4da3-a825-509bb5eba8f2 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:49.712274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f299e8-54f3-43bb-a657-eda619d16427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:49.712975 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f299e8-54f3-43bb-a657-eda619d16427 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:49.713290 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f299e8-54f3-43bb-a657-eda619d16427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:49.713606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f299e8-54f3-43bb-a657-eda619d16427 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:49.714513 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f299e8-54f3-43bb-a657-eda619d16427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:49.716303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f299e8-54f3-43bb-a657-eda619d16427 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:49.728045 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 949/1904] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:49.733527 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5188da1f-08d8-4ab3-bf7e-c91d84098818 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:49.754167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5188da1f-08d8-4ab3-bf7e-c91d84098818 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:49.762568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef294395-2639-4245-a4dd-3910c4c4a024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:49.764260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef294395-2639-4245-a4dd-3910c4c4a024 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:49.764544 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef294395-2639-4245-a4dd-3910c4c4a024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:49.764937 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef294395-2639-4245-a4dd-3910c4c4a024 admin admin] PATH_INFO: `/v3/projects/ae26dee45b8944d29b16987eaad57341/users/e5394911fba64277a4e0360808a47878/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:49.781337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef294395-2639-4245-a4dd-3910c4c4a024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae26dee45b8944d29b16987eaad57341, user_id=e5394911fba64277a4e0360808a47878, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:49.784071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef294395-2639-4245-a4dd-3910c4c4a024 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:49.799604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 956/1905] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:49 2022] PUT /identity/v3/projects/ae26dee45b8944d29b16987eaad57341/users/e5394911fba64277a4e0360808a47878/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:49.811253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984f2ff-a7be-46f9-ba52-36afb6a24e9c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:49.811591 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984f2ff-a7be-46f9-ba52-36afb6a24e9c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:49.813143 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984f2ff-a7be-46f9-ba52-36afb6a24e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:49.839680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9984f2ff-a7be-46f9-ba52-36afb6a24e9c None None] MFA Rules not processed for user `e5394911fba64277a4e0360808a47878`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:49.841975 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9984f2ff-a7be-46f9-ba52-36afb6a24e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:49.870617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:49.937205 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 950/1906] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:49.938448 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef294395-2639-4245-a4dd-3910c4c4a024 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:50.038873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef294395-2639-4245-a4dd-3910c4c4a024 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:50.044614 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0828cb-0686-4290-9731-1dd985ad60e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:50.045677 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0828cb-0686-4290-9731-1dd985ad60e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:50.045972 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0828cb-0686-4290-9731-1dd985ad60e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:50.046257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0828cb-0686-4290-9731-1dd985ad60e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:50.046960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0828cb-0686-4290-9731-1dd985ad60e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:50.049120 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0828cb-0686-4290-9731-1dd985ad60e0 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:50.082971 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 957/1907] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:49 2022] POST /identity/v3/projects => generated 381 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:50.086989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984f2ff-a7be-46f9-ba52-36afb6a24e9c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:50.105597 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984f2ff-a7be-46f9-ba52-36afb6a24e9c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:50.126260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0828cb-0686-4290-9731-1dd985ad60e0 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:50.128899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-9b0828cb-0686-4290-9731-1dd985ad60e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:50.199698 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e966ec-486e-4646-80bb-6f3fb04f1053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:50.200690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e966ec-486e-4646-80bb-6f3fb04f1053 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:50.201158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e966ec-486e-4646-80bb-6f3fb04f1053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:50.201545 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e966ec-486e-4646-80bb-6f3fb04f1053 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:50.202295 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e966ec-486e-4646-80bb-6f3fb04f1053 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:50.204338 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e966ec-486e-4646-80bb-6f3fb04f1053 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:50.235669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 951/1908] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:50 2022] POST /identity/v3/users => generated 297 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:50.240117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e966ec-486e-4646-80bb-6f3fb04f1053 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:50.245685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0828cb-0686-4290-9731-1dd985ad60e0 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:50.254511 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e966ec-486e-4646-80bb-6f3fb04f1053 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:50.260870 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e65f1-2469-4d8b-89ab-3cbc5faa7b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:50.261856 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e65f1-2469-4d8b-89ab-3cbc5faa7b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:50.261968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afe968-8c4b-4a72-ba8d-ff398c3a93af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:50.262211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e65f1-2469-4d8b-89ab-3cbc5faa7b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:50.262500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e65f1-2469-4d8b-89ab-3cbc5faa7b9a admin admin] PATH_INFO: `/v3/users/44e52bd61f8241dabb1b81fb5ece491f` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:50.263453 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afe968-8c4b-4a72-ba8d-ff398c3a93af admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:50.263953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afe968-8c4b-4a72-ba8d-ff398c3a93af admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:50.264354 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afe968-8c4b-4a72-ba8d-ff398c3a93af admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:50.265141 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afe968-8c4b-4a72-ba8d-ff398c3a93af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:50.266913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afe968-8c4b-4a72-ba8d-ff398c3a93af admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:50.273040 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e65f1-2469-4d8b-89ab-3cbc5faa7b9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=44e52bd61f8241dabb1b81fb5ece491f)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:50.275305 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e65f1-2469-4d8b-89ab-3cbc5faa7b9a admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:50.279910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 952/1909] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:50.284593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afe968-8c4b-4a72-ba8d-ff398c3a93af admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:50.301507 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afe968-8c4b-4a72-ba8d-ff398c3a93af admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:50.307911 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c0cf0-bc60-487b-a4f1-70b42a18d72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:50.309287 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c0cf0-bc60-487b-a4f1-70b42a18d72f admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:50.309798 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c0cf0-bc60-487b-a4f1-70b42a18d72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:50.310251 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c0cf0-bc60-487b-a4f1-70b42a18d72f admin admin] PATH_INFO: `/v3/projects/173470a38dbb44ec8da0f1d7fbcb496b/users/d78bf3bc3fd94334ae142e4a2a85f785/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:50.343967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c0cf0-bc60-487b-a4f1-70b42a18d72f admin admin] RBAC: Authorizing `identity:create_grant(project_id=173470a38dbb44ec8da0f1d7fbcb496b, user_id=d78bf3bc3fd94334ae142e4a2a85f785, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:50.348724 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c0cf0-bc60-487b-a4f1-70b42a18d72f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:50.368835 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:50.380296 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 953/1910] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:50 2022] PUT /identity/v3/projects/173470a38dbb44ec8da0f1d7fbcb496b/users/d78bf3bc3fd94334ae142e4a2a85f785/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:50.384872 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c0cf0-bc60-487b-a4f1-70b42a18d72f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:50.389213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-a54e65f1-2469-4d8b-89ab-3cbc5faa7b9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44e52bd61f8241dabb1b81fb5ece491f'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:50.400915 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-a54e65f1-2469-4d8b-89ab-3cbc5faa7b9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44e52bd61f8241dabb1b81fb5ece491f'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:50.413215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-a54e65f1-2469-4d8b-89ab-3cbc5faa7b9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44e52bd61f8241dabb1b81fb5ece491f'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:50.413956 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-a54e65f1-2469-4d8b-89ab-3cbc5faa7b9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44e52bd61f8241dabb1b81fb5ece491f'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:50.429015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-a54e65f1-2469-4d8b-89ab-3cbc5faa7b9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44e52bd61f8241dabb1b81fb5ece491f'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:50.438045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 958/1911] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:50 2022] DELETE /identity/v3/users/44e52bd61f8241dabb1b81fb5ece491f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:50.442404 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ca214-5dd0-4d08-a128-c1d81af8d434 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:50.442756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ca214-5dd0-4d08-a128-c1d81af8d434 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:50.443040 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ca214-5dd0-4d08-a128-c1d81af8d434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:50.461624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-6d5ca214-5dd0-4d08-a128-c1d81af8d434 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:50.462845 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-6d5ca214-5dd0-4d08-a128-c1d81af8d434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:50.466510 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c0cf0-bc60-487b-a4f1-70b42a18d72f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:50.486742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:50.531435 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 959/1912] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:38:50.538311 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c21b93-215b-41f3-8bc9-76d0e459ca27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:50.539130 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c21b93-215b-41f3-8bc9-76d0e459ca27 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:50.539490 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c21b93-215b-41f3-8bc9-76d0e459ca27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:50.539907 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c21b93-215b-41f3-8bc9-76d0e459ca27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:50.540896 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ca214-5dd0-4d08-a128-c1d81af8d434 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:50.541019 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c21b93-215b-41f3-8bc9-76d0e459ca27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:50.542708 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c21b93-215b-41f3-8bc9-76d0e459ca27 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:50.542892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-6d5ca214-5dd0-4d08-a128-c1d81af8d434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:50.558368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 954/1913] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:38:50.560621 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c21b93-215b-41f3-8bc9-76d0e459ca27 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:50.562957 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-53c21b93-215b-41f3-8bc9-76d0e459ca27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:50.577852 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c21b93-215b-41f3-8bc9-76d0e459ca27 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:50.586663 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f0ad3-30b0-4f6f-8bb4-26b116a286aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:50.588601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f0ad3-30b0-4f6f-8bb4-26b116a286aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:50.588601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f0ad3-30b0-4f6f-8bb4-26b116a286aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:50.589167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f0ad3-30b0-4f6f-8bb4-26b116a286aa admin admin] PATH_INFO: `/v3/projects/173470a38dbb44ec8da0f1d7fbcb496b/users/d78bf3bc3fd94334ae142e4a2a85f785/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:50.593002 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f0ad3-30b0-4f6f-8bb4-26b116a286aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=173470a38dbb44ec8da0f1d7fbcb496b, user_id=d78bf3bc3fd94334ae142e4a2a85f785, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:50.595820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f0ad3-30b0-4f6f-8bb4-26b116a286aa admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:50.610925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 955/1914] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:50 2022] PUT /identity/v3/projects/173470a38dbb44ec8da0f1d7fbcb496b/users/d78bf3bc3fd94334ae142e4a2a85f785/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:50.622738 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef96c8-ec60-4389-b158-44fc74e87cb6 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:50.623100 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef96c8-ec60-4389-b158-44fc74e87cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:50.623568 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef96c8-ec60-4389-b158-44fc74e87cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:50.641007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ca214-5dd0-4d08-a128-c1d81af8d434 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:50.674238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-31ef96c8-ec60-4389-b158-44fc74e87cb6 None None] MFA Rules not processed for user `d78bf3bc3fd94334ae142e4a2a85f785`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:50.676998 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-31ef96c8-ec60-4389-b158-44fc74e87cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:50.714037 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a334b-34c0-4e45-8618-e376b1846d35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:50.714723 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a334b-34c0-4e45-8618-e376b1846d35 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:50.716994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a334b-34c0-4e45-8618-e376b1846d35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:50.717368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a334b-34c0-4e45-8618-e376b1846d35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:50.769433 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:50.821968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 956/1915] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:50.832454 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef96c8-ec60-4389-b158-44fc74e87cb6 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:50.834319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-31ef96c8-ec60-4389-b158-44fc74e87cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:50.848957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a334b-34c0-4e45-8618-e376b1846d35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:50.853640 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a334b-34c0-4e45-8618-e376b1846d35 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:50.874171 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 960/1916] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:51.001195 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef96c8-ec60-4389-b158-44fc74e87cb6 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:51.005391 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0c23f-17a2-48c0-a5f8-1758f42b34f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:51.006298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0c23f-17a2-48c0-a5f8-1758f42b34f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:51.006589 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0c23f-17a2-48c0-a5f8-1758f42b34f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:51.006868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0c23f-17a2-48c0-a5f8-1758f42b34f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:51.031475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f0c23f-17a2-48c0-a5f8-1758f42b34f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:51.034802 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f0c23f-17a2-48c0-a5f8-1758f42b34f8 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:51.059152 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 957/1917] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:51.234806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a334b-34c0-4e45-8618-e376b1846d35 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:51.244530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a334b-34c0-4e45-8618-e376b1846d35 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:51.249981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133aa495-bbcd-40fe-a0fc-940377fb84ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:51.250645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133aa495-bbcd-40fe-a0fc-940377fb84ac service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:51.250926 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133aa495-bbcd-40fe-a0fc-940377fb84ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:51.251200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133aa495-bbcd-40fe-a0fc-940377fb84ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:51.253508 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-133aa495-bbcd-40fe-a0fc-940377fb84ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:51.335645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133aa495-bbcd-40fe-a0fc-940377fb84ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:51.337130 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133aa495-bbcd-40fe-a0fc-940377fb84ac service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:51.349048 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 961/1918] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:51.357413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0c23f-17a2-48c0-a5f8-1758f42b34f8 service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:51.359737 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-31f0c23f-17a2-48c0-a5f8-1758f42b34f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:51.410309 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0c23f-17a2-48c0-a5f8-1758f42b34f8 service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:51.422110 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8faa145-198d-48b9-887d-781c43368a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:51.422998 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8faa145-198d-48b9-887d-781c43368a1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:51.423278 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8faa145-198d-48b9-887d-781c43368a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:51.423560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8faa145-198d-48b9-887d-781c43368a1f admin admin] PATH_INFO: `/v3/projects/2dce49332f3d4ce6a2976f653418add4` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:51.430819 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8faa145-198d-48b9-887d-781c43368a1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dce49332f3d4ce6a2976f653418add4)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:51.432790 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8faa145-198d-48b9-887d-781c43368a1f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:51.529015 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:51.567080 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a8faa145-198d-48b9-887d-781c43368a1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dce49332f3d4ce6a2976f653418add4', 'request_id': 'req-a8faa145-198d-48b9-887d-781c43368a1f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:51.579648 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a8faa145-198d-48b9-887d-781c43368a1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dce49332f3d4ce6a2976f653418add4', 'request_id': 'req-a8faa145-198d-48b9-887d-781c43368a1f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:51.591860 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a8faa145-198d-48b9-887d-781c43368a1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dce49332f3d4ce6a2976f653418add4', 'request_id': 'req-a8faa145-198d-48b9-887d-781c43368a1f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:51.626159 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a8faa145-198d-48b9-887d-781c43368a1f admin admin] The token cache is being invalidate because project 2dce49332f3d4ce6a2976f653418add4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:51.626466 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a8faa145-198d-48b9-887d-781c43368a1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:51.628254 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a8faa145-198d-48b9-887d-781c43368a1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:51.631840 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 958/1919] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:51 2022] DELETE /identity/v3/projects/2dce49332f3d4ce6a2976f653418add4 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:38:51.637184 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0766a8-676a-4e01-bbc9-48d7ca597a44 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:51.637472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0766a8-676a-4e01-bbc9-48d7ca597a44 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:51.638681 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0766a8-676a-4e01-bbc9-48d7ca597a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:51.673701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-6f0766a8-676a-4e01-bbc9-48d7ca597a44 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:51.686656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-6f0766a8-676a-4e01-bbc9-48d7ca597a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:51.767300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 962/1920] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:51.776515 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59793c31-d87f-4354-810f-7a6b69d9e3fb None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:51.777061 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59793c31-d87f-4354-810f-7a6b69d9e3fb None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:51.777449 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59793c31-d87f-4354-810f-7a6b69d9e3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:51.799079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-59793c31-d87f-4354-810f-7a6b69d9e3fb None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:51.800461 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-59793c31-d87f-4354-810f-7a6b69d9e3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:51.814848 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 959/1921] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:51.818940 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0766a8-676a-4e01-bbc9-48d7ca597a44 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:51.833754 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0766a8-676a-4e01-bbc9-48d7ca597a44 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:51.839373 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2e838-a6b8-405d-9252-a21cd918447c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:51.840187 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2e838-a6b8-405d-9252-a21cd918447c admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:51.840579 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2e838-a6b8-405d-9252-a21cd918447c admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:51.840989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2e838-a6b8-405d-9252-a21cd918447c admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:51.841824 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d2e838-a6b8-405d-9252-a21cd918447c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:38:51.842137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d2e838-a6b8-405d-9252-a21cd918447c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:51.843768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d2e838-a6b8-405d-9252-a21cd918447c admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:51.860060 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 963/1922] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:38:51.864682 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59793c31-d87f-4354-810f-7a6b69d9e3fb None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:51.877944 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59793c31-d87f-4354-810f-7a6b69d9e3fb None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:51.894868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eeb841-61b9-41ad-800c-15331e097f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:51.895860 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eeb841-61b9-41ad-800c-15331e097f4d admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:51.896145 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eeb841-61b9-41ad-800c-15331e097f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:51.896433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eeb841-61b9-41ad-800c-15331e097f4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:51.897140 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eeb841-61b9-41ad-800c-15331e097f4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:51.899133 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eeb841-61b9-41ad-800c-15331e097f4d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:51.932423 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 960/1923] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:51 2022] POST /identity/v3/projects => generated 367 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:51.937629 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2e838-a6b8-405d-9252-a21cd918447c admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:51.949826 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2e838-a6b8-405d-9252-a21cd918447c admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:52.002353 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:52.016188 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef426db9-2d3c-4c7c-82a1-0a4173d13be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:52.017008 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef426db9-2d3c-4c7c-82a1-0a4173d13be2 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:52.017291 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef426db9-2d3c-4c7c-82a1-0a4173d13be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:52.017567 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef426db9-2d3c-4c7c-82a1-0a4173d13be2 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:52.018264 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef426db9-2d3c-4c7c-82a1-0a4173d13be2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:52.021213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef426db9-2d3c-4c7c-82a1-0a4173d13be2 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:52.048431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 964/1924] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:51 2022] POST /identity/v3/users => generated 284 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:38:52.049648 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eeb841-61b9-41ad-800c-15331e097f4d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:52.060779 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eeb841-61b9-41ad-800c-15331e097f4d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:52.065670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac57bf8-9483-41d5-8af3-b5427fa1340f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:52.066334 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac57bf8-9483-41d5-8af3-b5427fa1340f admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:52.066616 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac57bf8-9483-41d5-8af3-b5427fa1340f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:52.066891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac57bf8-9483-41d5-8af3-b5427fa1340f admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:52.067614 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac57bf8-9483-41d5-8af3-b5427fa1340f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:52.069279 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac57bf8-9483-41d5-8af3-b5427fa1340f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:52.080299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 961/1925] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:38:52.084247 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef426db9-2d3c-4c7c-82a1-0a4173d13be2 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:52.095283 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef426db9-2d3c-4c7c-82a1-0a4173d13be2 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:52.100478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d1c5b-e7e4-44ee-97f1-fed73bfbe76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:52.101148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d1c5b-e7e4-44ee-97f1-fed73bfbe76a admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:52.101427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d1c5b-e7e4-44ee-97f1-fed73bfbe76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:52.101701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d1c5b-e7e4-44ee-97f1-fed73bfbe76a admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:52.102453 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d1c5b-e7e4-44ee-97f1-fed73bfbe76a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:52.104109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d1c5b-e7e4-44ee-97f1-fed73bfbe76a admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:52.114643 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 965/1926] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:52.119963 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac57bf8-9483-41d5-8af3-b5427fa1340f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:52.129333 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac57bf8-9483-41d5-8af3-b5427fa1340f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:52.134238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a861f-75f3-43ea-954a-0df3009be8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:52.135113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a861f-75f3-43ea-954a-0df3009be8e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:52.135397 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a861f-75f3-43ea-954a-0df3009be8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:52.135662 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a861f-75f3-43ea-954a-0df3009be8e9 admin admin] PATH_INFO: `/v3/projects/b58baaa33f7940aab10e76c4423596d2/users/c57cf1e43a9f49edac6e006444eb73c9/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:52.151954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a861f-75f3-43ea-954a-0df3009be8e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b58baaa33f7940aab10e76c4423596d2, user_id=c57cf1e43a9f49edac6e006444eb73c9, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:52.155275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a861f-75f3-43ea-954a-0df3009be8e9 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:52.170563 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 962/1927] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:52 2022] PUT /identity/v3/projects/b58baaa33f7940aab10e76c4423596d2/users/c57cf1e43a9f49edac6e006444eb73c9/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:52.180493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062880c-ea7f-4910-808f-0544009be75b None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:52.180791 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062880c-ea7f-4910-808f-0544009be75b None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:52.181061 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062880c-ea7f-4910-808f-0544009be75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:52.208613 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-1062880c-ea7f-4910-808f-0544009be75b None None] MFA Rules not processed for user `c57cf1e43a9f49edac6e006444eb73c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:52.209759 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-1062880c-ea7f-4910-808f-0544009be75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:52.282680 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 966/1928] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:52 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:52.295344 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a861f-75f3-43ea-954a-0df3009be8e9 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:52.297105 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f82a861f-75f3-43ea-954a-0df3009be8e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:52.357043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a861f-75f3-43ea-954a-0df3009be8e9 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:52.362042 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95656ac7-8d1b-4f50-96f8-355031887794 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a67e007fc314b34aac2471a7c20f240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:52.362042 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95656ac7-8d1b-4f50-96f8-355031887794 service glance] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:52.362042 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95656ac7-8d1b-4f50-96f8-355031887794 service glance] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:52.362042 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95656ac7-8d1b-4f50-96f8-355031887794 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:52.370525 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95656ac7-8d1b-4f50-96f8-355031887794 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:52.371995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95656ac7-8d1b-4f50-96f8-355031887794 service glance] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:52.382415 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 963/1929] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:38:54.250791 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062880c-ea7f-4910-808f-0544009be75b None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:54.252412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-1062880c-ea7f-4910-808f-0544009be75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:54.334800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062880c-ea7f-4910-808f-0544009be75b None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:54.339738 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0975c9-fa32-4dc1-b1d4-3169402f8e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:54.340727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0975c9-fa32-4dc1-b1d4-3169402f8e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:54.341315 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0975c9-fa32-4dc1-b1d4-3169402f8e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:54.341966 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0975c9-fa32-4dc1-b1d4-3169402f8e93 admin admin] PATH_INFO: `/v3/users/e5394911fba64277a4e0360808a47878` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:54.343324 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0975c9-fa32-4dc1-b1d4-3169402f8e93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5394911fba64277a4e0360808a47878)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:54.345422 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0975c9-fa32-4dc1-b1d4-3169402f8e93 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:54.407653 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-ff0975c9-fa32-4dc1-b1d4-3169402f8e93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5394911fba64277a4e0360808a47878'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:54.424392 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-ff0975c9-fa32-4dc1-b1d4-3169402f8e93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5394911fba64277a4e0360808a47878'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:54.428275 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:54.432746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-ff0975c9-fa32-4dc1-b1d4-3169402f8e93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5394911fba64277a4e0360808a47878'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:54.433517 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-ff0975c9-fa32-4dc1-b1d4-3169402f8e93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5394911fba64277a4e0360808a47878'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:54.449925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-ff0975c9-fa32-4dc1-b1d4-3169402f8e93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5394911fba64277a4e0360808a47878'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:54.456689 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 967/1930] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:54 2022] DELETE /identity/v3/users/e5394911fba64277a4e0360808a47878 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:54.461487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf30e6-8d12-416b-814a-7dc8c11e05a3 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:54.462017 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf30e6-8d12-416b-814a-7dc8c11e05a3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:54.462158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf30e6-8d12-416b-814a-7dc8c11e05a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:54.480026 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-79cf30e6-8d12-416b-814a-7dc8c11e05a3 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:54.482075 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-79cf30e6-8d12-416b-814a-7dc8c11e05a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:54.532196 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:54.552591 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 964/1931] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:54.567751 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0975c9-fa32-4dc1-b1d4-3169402f8e93 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:54.569214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ff0975c9-fa32-4dc1-b1d4-3169402f8e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:54.635369 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0975c9-fa32-4dc1-b1d4-3169402f8e93 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:54.640541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97772bef-7374-4b05-a0bc-f5e2cd9a0674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:54.641255 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97772bef-7374-4b05-a0bc-f5e2cd9a0674 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:54.641625 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97772bef-7374-4b05-a0bc-f5e2cd9a0674 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:54.641924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97772bef-7374-4b05-a0bc-f5e2cd9a0674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:54.651645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97772bef-7374-4b05-a0bc-f5e2cd9a0674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:54.653091 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97772bef-7374-4b05-a0bc-f5e2cd9a0674 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:54.666554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 968/1932] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:38:55.162368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf30e6-8d12-416b-814a-7dc8c11e05a3 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:55.164868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-79cf30e6-8d12-416b-814a-7dc8c11e05a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:55.191356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf30e6-8d12-416b-814a-7dc8c11e05a3 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:55.212627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42f637-be0b-46ca-a60c-5739c266666c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:55.214997 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42f637-be0b-46ca-a60c-5739c266666c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:55.219911 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42f637-be0b-46ca-a60c-5739c266666c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:55.220284 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42f637-be0b-46ca-a60c-5739c266666c admin admin] PATH_INFO: `/v3/projects/ae26dee45b8944d29b16987eaad57341` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:55.222618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d42f637-be0b-46ca-a60c-5739c266666c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae26dee45b8944d29b16987eaad57341)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:55.229875 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d42f637-be0b-46ca-a60c-5739c266666c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:55.383349 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-6d42f637-be0b-46ca-a60c-5739c266666c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae26dee45b8944d29b16987eaad57341', 'request_id': 'req-6d42f637-be0b-46ca-a60c-5739c266666c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:55.394952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-6d42f637-be0b-46ca-a60c-5739c266666c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae26dee45b8944d29b16987eaad57341', 'request_id': 'req-6d42f637-be0b-46ca-a60c-5739c266666c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:55.411564 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-6d42f637-be0b-46ca-a60c-5739c266666c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae26dee45b8944d29b16987eaad57341', 'request_id': 'req-6d42f637-be0b-46ca-a60c-5739c266666c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:55.433386 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-6d42f637-be0b-46ca-a60c-5739c266666c admin admin] The token cache is being invalidate because project ae26dee45b8944d29b16987eaad57341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:55.433905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-6d42f637-be0b-46ca-a60c-5739c266666c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:55.434885 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-6d42f637-be0b-46ca-a60c-5739c266666c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:55.436703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 965/1933] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:55 2022] DELETE /identity/v3/projects/ae26dee45b8944d29b16987eaad57341 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:55.441523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97772bef-7374-4b05-a0bc-f5e2cd9a0674 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:55.454434 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97772bef-7374-4b05-a0bc-f5e2cd9a0674 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:55.542429 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42f637-be0b-46ca-a60c-5739c266666c admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:55.545404 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6d42f637-be0b-46ca-a60c-5739c266666c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:55.552500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46999ccc-f673-4798-8c6c-c55cf31ef206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:55.553773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46999ccc-f673-4798-8c6c-c55cf31ef206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:55.555041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46999ccc-f673-4798-8c6c-c55cf31ef206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:55.555327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46999ccc-f673-4798-8c6c-c55cf31ef206 admin admin] PATH_INFO: `/v3/users/d78bf3bc3fd94334ae142e4a2a85f785` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:55.557092 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46999ccc-f673-4798-8c6c-c55cf31ef206 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d78bf3bc3fd94334ae142e4a2a85f785)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:55.559581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46999ccc-f673-4798-8c6c-c55cf31ef206 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:55.627210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-46999ccc-f673-4798-8c6c-c55cf31ef206 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd78bf3bc3fd94334ae142e4a2a85f785'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:55.629430 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42f637-be0b-46ca-a60c-5739c266666c admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:55.634316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-46999ccc-f673-4798-8c6c-c55cf31ef206 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd78bf3bc3fd94334ae142e4a2a85f785'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:55.637323 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ad65c-5f5f-4bea-bc19-02e8623d42ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:55.637474 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ad65c-5f5f-4bea-bc19-02e8623d42ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:55.638595 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ad65c-5f5f-4bea-bc19-02e8623d42ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:55.638865 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ad65c-5f5f-4bea-bc19-02e8623d42ce admin admin] PATH_INFO: `/v3/users/c47d8ba602c54febac5f6203f9517351` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:55.640466 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ad65c-5f5f-4bea-bc19-02e8623d42ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=c47d8ba602c54febac5f6203f9517351)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:55.642758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ad65c-5f5f-4bea-bc19-02e8623d42ce admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:55.644642 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-46999ccc-f673-4798-8c6c-c55cf31ef206 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd78bf3bc3fd94334ae142e4a2a85f785'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:55.645725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-46999ccc-f673-4798-8c6c-c55cf31ef206 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd78bf3bc3fd94334ae142e4a2a85f785'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:55.663969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-46999ccc-f673-4798-8c6c-c55cf31ef206 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd78bf3bc3fd94334ae142e4a2a85f785'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:55.667205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:55.672980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 969/1934] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:55 2022] DELETE /identity/v3/users/d78bf3bc3fd94334ae142e4a2a85f785 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:38:55.701693 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-555ad65c-5f5f-4bea-bc19-02e8623d42ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c47d8ba602c54febac5f6203f9517351'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:55.712467 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-555ad65c-5f5f-4bea-bc19-02e8623d42ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c47d8ba602c54febac5f6203f9517351'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:55.713346 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-555ad65c-5f5f-4bea-bc19-02e8623d42ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c47d8ba602c54febac5f6203f9517351'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:55.714057 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-555ad65c-5f5f-4bea-bc19-02e8623d42ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c47d8ba602c54febac5f6203f9517351'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:55.728711 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-555ad65c-5f5f-4bea-bc19-02e8623d42ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c47d8ba602c54febac5f6203f9517351'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:55.740457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:55.749408 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 966/1935] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:55 2022] DELETE /identity/v3/users/c47d8ba602c54febac5f6203f9517351 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:55.980594 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46999ccc-f673-4798-8c6c-c55cf31ef206 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:55.989797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-46999ccc-f673-4798-8c6c-c55cf31ef206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:56.023436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ad65c-5f5f-4bea-bc19-02e8623d42ce admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:56.025120 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-555ad65c-5f5f-4bea-bc19-02e8623d42ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:56.110582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46999ccc-f673-4798-8c6c-c55cf31ef206 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:56.116022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea21e5-6770-477a-9886-5fea947554da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:56.116892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea21e5-6770-477a-9886-5fea947554da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:56.117197 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea21e5-6770-477a-9886-5fea947554da admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:56.117629 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea21e5-6770-477a-9886-5fea947554da admin admin] PATH_INFO: `/v3/projects/09aef592ea7042ecbc4687aafb0f2469` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:56.118813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea21e5-6770-477a-9886-5fea947554da admin admin] RBAC: Authorizing `identity:delete_project(project_id=09aef592ea7042ecbc4687aafb0f2469)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:56.120764 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea21e5-6770-477a-9886-5fea947554da admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:56.121928 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ad65c-5f5f-4bea-bc19-02e8623d42ce admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:56.128230 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce4a21-467f-4ca6-8d35-9536fac0a16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:56.129032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce4a21-467f-4ca6-8d35-9536fac0a16f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:56.129316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce4a21-467f-4ca6-8d35-9536fac0a16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:56.129592 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce4a21-467f-4ca6-8d35-9536fac0a16f admin admin] PATH_INFO: `/v3/projects/173470a38dbb44ec8da0f1d7fbcb496b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:56.130779 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce4a21-467f-4ca6-8d35-9536fac0a16f admin admin] RBAC: Authorizing `identity:delete_project(project_id=173470a38dbb44ec8da0f1d7fbcb496b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:56.132725 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce4a21-467f-4ca6-8d35-9536fac0a16f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:56.179133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-efea21e5-6770-477a-9886-5fea947554da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09aef592ea7042ecbc4687aafb0f2469', 'request_id': 'req-efea21e5-6770-477a-9886-5fea947554da'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:56.186267 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-efea21e5-6770-477a-9886-5fea947554da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09aef592ea7042ecbc4687aafb0f2469', 'request_id': 'req-efea21e5-6770-477a-9886-5fea947554da'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:56.196808 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-efea21e5-6770-477a-9886-5fea947554da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09aef592ea7042ecbc4687aafb0f2469', 'request_id': 'req-efea21e5-6770-477a-9886-5fea947554da'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:56.203634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a0ce4a21-467f-4ca6-8d35-9536fac0a16f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '173470a38dbb44ec8da0f1d7fbcb496b', 'request_id': 'req-a0ce4a21-467f-4ca6-8d35-9536fac0a16f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:56.212998 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a0ce4a21-467f-4ca6-8d35-9536fac0a16f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '173470a38dbb44ec8da0f1d7fbcb496b', 'request_id': 'req-a0ce4a21-467f-4ca6-8d35-9536fac0a16f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:56.213550 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-efea21e5-6770-477a-9886-5fea947554da admin admin] The token cache is being invalidate because project 09aef592ea7042ecbc4687aafb0f2469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:56.218878 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-efea21e5-6770-477a-9886-5fea947554da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:56.219534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-efea21e5-6770-477a-9886-5fea947554da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:56.222882 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 970/1936] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:55 2022] DELETE /identity/v3/projects/09aef592ea7042ecbc4687aafb0f2469 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:56.227138 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea21e5-6770-477a-9886-5fea947554da admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:56.227452 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a0ce4a21-467f-4ca6-8d35-9536fac0a16f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '173470a38dbb44ec8da0f1d7fbcb496b', 'request_id': 'req-a0ce4a21-467f-4ca6-8d35-9536fac0a16f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:56.229508 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-efea21e5-6770-477a-9886-5fea947554da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:56.243617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a0ce4a21-467f-4ca6-8d35-9536fac0a16f admin admin] The token cache is being invalidate because project 173470a38dbb44ec8da0f1d7fbcb496b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:56.243951 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a0ce4a21-467f-4ca6-8d35-9536fac0a16f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:56.245099 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a0ce4a21-467f-4ca6-8d35-9536fac0a16f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:56.253206 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 967/1937] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:56 2022] DELETE /identity/v3/projects/173470a38dbb44ec8da0f1d7fbcb496b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:38:56.254966 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b24a3f-3128-4f0e-aaf9-9734d9f30e86 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:56.255364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b24a3f-3128-4f0e-aaf9-9734d9f30e86 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:56.255538 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b24a3f-3128-4f0e-aaf9-9734d9f30e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:56.276166 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-50b24a3f-3128-4f0e-aaf9-9734d9f30e86 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:56.277677 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-50b24a3f-3128-4f0e-aaf9-9734d9f30e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:56.359361 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea21e5-6770-477a-9886-5fea947554da admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:56.423429 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:56.433497 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 968/1938] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:56.444081 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a2b64-610e-4b6a-a395-c1bb6fe2fdc7 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:56.445014 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a2b64-610e-4b6a-a395-c1bb6fe2fdc7 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:56.445927 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a2b64-610e-4b6a-a395-c1bb6fe2fdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:56.463994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edcd3d-75d0-4f3f-88ca-c576f950c42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:56.465599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edcd3d-75d0-4f3f-88ca-c576f950c42c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:56.466998 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edcd3d-75d0-4f3f-88ca-c576f950c42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:56.467473 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edcd3d-75d0-4f3f-88ca-c576f950c42c admin admin] PATH_INFO: `/v3/users/26243b992121478595db9a6de7950e9a` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:56.469381 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46edcd3d-75d0-4f3f-88ca-c576f950c42c admin admin] RBAC: Authorizing `identity:delete_user(user_id=26243b992121478595db9a6de7950e9a)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:56.472154 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46edcd3d-75d0-4f3f-88ca-c576f950c42c admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:56.508683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c41a2b64-610e-4b6a-a395-c1bb6fe2fdc7 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:56.510146 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c41a2b64-610e-4b6a-a395-c1bb6fe2fdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:56.528392 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 969/1939] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:56.533729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a2b64-610e-4b6a-a395-c1bb6fe2fdc7 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:56.543158 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-46edcd3d-75d0-4f3f-88ca-c576f950c42c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26243b992121478595db9a6de7950e9a'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:56.548394 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a2b64-610e-4b6a-a395-c1bb6fe2fdc7 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:56.551288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-46edcd3d-75d0-4f3f-88ca-c576f950c42c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26243b992121478595db9a6de7950e9a'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:56.553538 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92546079-142e-41df-bb4d-b12a49b02d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:56.554378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92546079-142e-41df-bb4d-b12a49b02d9d admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:56.554905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92546079-142e-41df-bb4d-b12a49b02d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:56.555330 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92546079-142e-41df-bb4d-b12a49b02d9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:56.556250 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92546079-142e-41df-bb4d-b12a49b02d9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:38:56.556705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92546079-142e-41df-bb4d-b12a49b02d9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:56.558489 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92546079-142e-41df-bb4d-b12a49b02d9d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:56.570833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-46edcd3d-75d0-4f3f-88ca-c576f950c42c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26243b992121478595db9a6de7950e9a'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:56.571907 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-46edcd3d-75d0-4f3f-88ca-c576f950c42c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26243b992121478595db9a6de7950e9a'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:56.580596 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 970/1940] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:38:56.585297 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92546079-142e-41df-bb4d-b12a49b02d9d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:56.592211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-46edcd3d-75d0-4f3f-88ca-c576f950c42c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26243b992121478595db9a6de7950e9a'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:56.597875 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92546079-142e-41df-bb4d-b12a49b02d9d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:56.609360 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 971/1941] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:56 2022] DELETE /identity/v3/users/26243b992121478595db9a6de7950e9a => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:38:56.609694 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff5c64-6a1f-466e-89e1-2a00f4cd2e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:56.610989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff5c64-6a1f-466e-89e1-2a00f4cd2e1b admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:56.610989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff5c64-6a1f-466e-89e1-2a00f4cd2e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:56.611154 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff5c64-6a1f-466e-89e1-2a00f4cd2e1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:56.611830 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff5c64-6a1f-466e-89e1-2a00f4cd2e1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:56.614398 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff5c64-6a1f-466e-89e1-2a00f4cd2e1b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:56.626313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:56.662558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 971/1942] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:56 2022] POST /identity/v3/projects => generated 383 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:38:56.666922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edcd3d-75d0-4f3f-88ca-c576f950c42c admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:56.673001 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-46edcd3d-75d0-4f3f-88ca-c576f950c42c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:56.762246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edcd3d-75d0-4f3f-88ca-c576f950c42c admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:56.865328 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed97d9-2e7f-4718-b260-bac84702f3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:56.866324 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed97d9-2e7f-4718-b260-bac84702f3b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:56.866617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed97d9-2e7f-4718-b260-bac84702f3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:56.866905 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed97d9-2e7f-4718-b260-bac84702f3b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:56.867599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed97d9-2e7f-4718-b260-bac84702f3b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:56.869578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed97d9-2e7f-4718-b260-bac84702f3b4 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:56.883077 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:56.902329 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff5c64-6a1f-466e-89e1-2a00f4cd2e1b admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:56.904008 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 972/1943] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:56 2022] POST /identity/v3/users => generated 292 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:56.918184 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed97d9-2e7f-4718-b260-bac84702f3b4 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:56.921567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff5c64-6a1f-466e-89e1-2a00f4cd2e1b admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:56.922747 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-48ed97d9-2e7f-4718-b260-bac84702f3b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:56.928714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcacc183-61d4-4b91-9f6d-6400fa8b2cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:56.929455 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcacc183-61d4-4b91-9f6d-6400fa8b2cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:56.929749 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcacc183-61d4-4b91-9f6d-6400fa8b2cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:56.930031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcacc183-61d4-4b91-9f6d-6400fa8b2cf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:56.930818 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcacc183-61d4-4b91-9f6d-6400fa8b2cf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:56.934827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcacc183-61d4-4b91-9f6d-6400fa8b2cf4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:56.948124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed97d9-2e7f-4718-b260-bac84702f3b4 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:56.958076 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866df63-faf5-4d74-a754-676ed137cbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:56.958438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866df63-faf5-4d74-a754-676ed137cbb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:56.958881 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866df63-faf5-4d74-a754-676ed137cbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:56.959376 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866df63-faf5-4d74-a754-676ed137cbb6 admin admin] PATH_INFO: `/v3/projects/9afffa6608c94d81a94cfc3a7c452a7c` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:56.961937 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866df63-faf5-4d74-a754-676ed137cbb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9afffa6608c94d81a94cfc3a7c452a7c)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:56.964881 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 972/1944] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:38:56.965308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866df63-faf5-4d74-a754-676ed137cbb6 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:56.966784 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcacc183-61d4-4b91-9f6d-6400fa8b2cf4 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:56.994780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcacc183-61d4-4b91-9f6d-6400fa8b2cf4 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:57.026623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-5866df63-faf5-4d74-a754-676ed137cbb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9afffa6608c94d81a94cfc3a7c452a7c', 'request_id': 'req-5866df63-faf5-4d74-a754-676ed137cbb6'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:57.030639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4251f6-3ddc-4e28-a3ec-80c1510aa711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:57.031381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4251f6-3ddc-4e28-a3ec-80c1510aa711 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:57.031929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4251f6-3ddc-4e28-a3ec-80c1510aa711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:57.032081 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4251f6-3ddc-4e28-a3ec-80c1510aa711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:57.032818 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4251f6-3ddc-4e28-a3ec-80c1510aa711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:57.035746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-5866df63-faf5-4d74-a754-676ed137cbb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9afffa6608c94d81a94cfc3a7c452a7c', 'request_id': 'req-5866df63-faf5-4d74-a754-676ed137cbb6'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:57.037471 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4251f6-3ddc-4e28-a3ec-80c1510aa711 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:57.046977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-5866df63-faf5-4d74-a754-676ed137cbb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9afffa6608c94d81a94cfc3a7c452a7c', 'request_id': 'req-5866df63-faf5-4d74-a754-676ed137cbb6'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:57.064029 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 973/1945] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:57.067220 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4251f6-3ddc-4e28-a3ec-80c1510aa711 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:57.077143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-5866df63-faf5-4d74-a754-676ed137cbb6 admin admin] The token cache is being invalidate because project 9afffa6608c94d81a94cfc3a7c452a7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:57.077444 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-5866df63-faf5-4d74-a754-676ed137cbb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:57.080358 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-5866df63-faf5-4d74-a754-676ed137cbb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:57.083502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 973/1946] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:56 2022] DELETE /identity/v3/projects/9afffa6608c94d81a94cfc3a7c452a7c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:57.089237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866412d-64a6-4196-8351-5f553e93a3d0 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:57.089526 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866412d-64a6-4196-8351-5f553e93a3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:57.089804 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866412d-64a6-4196-8351-5f553e93a3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:57.095372 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4251f6-3ddc-4e28-a3ec-80c1510aa711 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:57.112904 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-2866412d-64a6-4196-8351-5f553e93a3d0 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:57.114239 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-2866412d-64a6-4196-8351-5f553e93a3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:57.174589 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451cbd13-6315-4716-b97a-f0b732cf1ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:57.175394 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cbd13-6315-4716-b97a-f0b732cf1ce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:57.175671 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cbd13-6315-4716-b97a-f0b732cf1ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:57.175976 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cbd13-6315-4716-b97a-f0b732cf1ce9 admin admin] PATH_INFO: `/v3/projects/cb6299abef70443b9a5d522f19b5fb5e/users/48aeafa9123a43329cfe85ef74dd62f9/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:57.193079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451cbd13-6315-4716-b97a-f0b732cf1ce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6299abef70443b9a5d522f19b5fb5e, user_id=48aeafa9123a43329cfe85ef74dd62f9, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:57.195165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 974/1947] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:57.203871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451cbd13-6315-4716-b97a-f0b732cf1ce9 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:57.205142 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ac1a3-6b06-4f0e-a9d4-f9ca797890d2 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:57.205429 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ac1a3-6b06-4f0e-a9d4-f9ca797890d2 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:57.205702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ac1a3-6b06-4f0e-a9d4-f9ca797890d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:57.213593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 974/1948] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:57 2022] PUT /identity/v3/projects/cb6299abef70443b9a5d522f19b5fb5e/users/48aeafa9123a43329cfe85ef74dd62f9/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:57.224822 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-f20ac1a3-6b06-4f0e-a9d4-f9ca797890d2 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:57.225003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713716ca-91b9-465b-a5ac-5325dee529ca None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:57.225239 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713716ca-91b9-465b-a5ac-5325dee529ca None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:57.225521 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713716ca-91b9-465b-a5ac-5325dee529ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:57.226031 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-f20ac1a3-6b06-4f0e-a9d4-f9ca797890d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:57.258303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-713716ca-91b9-465b-a5ac-5325dee529ca None None] MFA Rules not processed for user `48aeafa9123a43329cfe85ef74dd62f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:57.259511 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-713716ca-91b9-465b-a5ac-5325dee529ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:57.303150 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 975/1949] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:38:57.307414 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ac1a3-6b06-4f0e-a9d4-f9ca797890d2 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:57.317261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ac1a3-6b06-4f0e-a9d4-f9ca797890d2 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:57.323092 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f2335-8a0e-4113-ab1b-efb3683db00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:57.323940 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f2335-8a0e-4113-ab1b-efb3683db00c admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:57.324254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f2335-8a0e-4113-ab1b-efb3683db00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:57.324541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f2335-8a0e-4113-ab1b-efb3683db00c admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:57.325892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f2335-8a0e-4113-ab1b-efb3683db00c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:38:57.326205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f2335-8a0e-4113-ab1b-efb3683db00c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:57.327867 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f2335-8a0e-4113-ab1b-efb3683db00c admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:57.339617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 975/1950] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:38:57.344926 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713716ca-91b9-465b-a5ac-5325dee529ca None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:57.345678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 976/1951] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:38:57.346867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-713716ca-91b9-465b-a5ac-5325dee529ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:57.350485 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f2335-8a0e-4113-ab1b-efb3683db00c admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:57.358778 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:57.365007 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713716ca-91b9-465b-a5ac-5325dee529ca None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:57.365650 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f2335-8a0e-4113-ab1b-efb3683db00c admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:57.371184 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb4b0f-7014-4788-8c67-96afc39c2ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:57.372615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb4b0f-7014-4788-8c67-96afc39c2ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:57.372905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb4b0f-7014-4788-8c67-96afc39c2ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:57.373745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb4b0f-7014-4788-8c67-96afc39c2ea2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:57.374591 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb4b0f-7014-4788-8c67-96afc39c2ea2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:57.375433 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f730676-03aa-4ad8-8a5e-1b7975d652bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:57.376323 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f730676-03aa-4ad8-8a5e-1b7975d652bb admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:57.376709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f730676-03aa-4ad8-8a5e-1b7975d652bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:57.377095 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f730676-03aa-4ad8-8a5e-1b7975d652bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:57.380261 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb4b0f-7014-4788-8c67-96afc39c2ea2 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:57.388509 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f730676-03aa-4ad8-8a5e-1b7975d652bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:57.388509 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f730676-03aa-4ad8-8a5e-1b7975d652bb admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:57.422407 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 976/1952] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:57 2022] POST /identity/v3/projects => generated 383 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:57.425897 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 977/1953] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:57 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:38:57.426931 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb4b0f-7014-4788-8c67-96afc39c2ea2 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:57.429719 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f730676-03aa-4ad8-8a5e-1b7975d652bb admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:57.446418 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:57.502618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb4b0f-7014-4788-8c67-96afc39c2ea2 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:57.503925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f730676-03aa-4ad8-8a5e-1b7975d652bb admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:57.508613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f7168-c4e7-40fd-b946-77b7fd63604e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:57.512747 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f7168-c4e7-40fd-b946-77b7fd63604e admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:57.513697 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f7168-c4e7-40fd-b946-77b7fd63604e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:57.513851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f7168-c4e7-40fd-b946-77b7fd63604e admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:57.515318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f7168-c4e7-40fd-b946-77b7fd63604e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:57.515938 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93269fff-30f7-45e6-984f-4335c7a3ec33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:57.516742 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93269fff-30f7-45e6-984f-4335c7a3ec33 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:57.517039 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93269fff-30f7-45e6-984f-4335c7a3ec33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:57.517483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93269fff-30f7-45e6-984f-4335c7a3ec33 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:57.518175 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93269fff-30f7-45e6-984f-4335c7a3ec33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:57.518307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f7168-c4e7-40fd-b946-77b7fd63604e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:57.520994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93269fff-30f7-45e6-984f-4335c7a3ec33 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:57.547120 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 977/1954] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:57 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:57.550058 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 978/1955] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:57 2022] POST /identity/v3/users => generated 286 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:57.553856 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f7168-c4e7-40fd-b946-77b7fd63604e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:57.555687 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93269fff-30f7-45e6-984f-4335c7a3ec33 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:57.565092 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f7168-c4e7-40fd-b946-77b7fd63604e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:57.571850 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93269fff-30f7-45e6-984f-4335c7a3ec33 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:57.573439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a37e2b-27f6-478f-87de-fd36c0f3de66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:57.574217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a37e2b-27f6-478f-87de-fd36c0f3de66 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:57.574539 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a37e2b-27f6-478f-87de-fd36c0f3de66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:57.574824 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a37e2b-27f6-478f-87de-fd36c0f3de66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:57.575761 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a37e2b-27f6-478f-87de-fd36c0f3de66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:57.577580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a37e2b-27f6-478f-87de-fd36c0f3de66 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:57.577708 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3a255-d701-4b9b-b951-366609be1eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:57.578397 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3a255-d701-4b9b-b951-366609be1eec admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:57.578754 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3a255-d701-4b9b-b951-366609be1eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:57.579201 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3a255-d701-4b9b-b951-366609be1eec admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:57.580089 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3a255-d701-4b9b-b951-366609be1eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:57.581899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3a255-d701-4b9b-b951-366609be1eec admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:57.588437 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 978/1956] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:57.593371 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a37e2b-27f6-478f-87de-fd36c0f3de66 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:57.599545 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 979/1957] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:57.601632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3a255-d701-4b9b-b951-366609be1eec admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:57.608782 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a37e2b-27f6-478f-87de-fd36c0f3de66 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:57.614562 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3a255-d701-4b9b-b951-366609be1eec admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:57.617194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31f1f3-d6bd-4ec2-98ea-6d13ba91349e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:57.617908 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31f1f3-d6bd-4ec2-98ea-6d13ba91349e admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:57.618220 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31f1f3-d6bd-4ec2-98ea-6d13ba91349e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:57.618498 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31f1f3-d6bd-4ec2-98ea-6d13ba91349e admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:57.619352 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31f1f3-d6bd-4ec2-98ea-6d13ba91349e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:57.620146 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583304c-0b38-4abb-b5b6-b931dee0f854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:57.620955 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583304c-0b38-4abb-b5b6-b931dee0f854 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:57.621152 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31f1f3-d6bd-4ec2-98ea-6d13ba91349e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:57.621248 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583304c-0b38-4abb-b5b6-b931dee0f854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:57.621504 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583304c-0b38-4abb-b5b6-b931dee0f854 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:57.622908 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583304c-0b38-4abb-b5b6-b931dee0f854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:57.624649 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583304c-0b38-4abb-b5b6-b931dee0f854 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:57.634354 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 979/1958] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:57.637596 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 980/1959] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:57.639344 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31f1f3-d6bd-4ec2-98ea-6d13ba91349e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:57.642107 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583304c-0b38-4abb-b5b6-b931dee0f854 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:57.651924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31f1f3-d6bd-4ec2-98ea-6d13ba91349e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:57.652729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583304c-0b38-4abb-b5b6-b931dee0f854 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:57.657153 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06a629-691c-437d-9064-c471df75d41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:57.657908 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816e32b-6885-426b-8c5d-1be76fbe4e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:57.658181 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06a629-691c-437d-9064-c471df75d41a admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:57.658527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06a629-691c-437d-9064-c471df75d41a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:57.660121 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06a629-691c-437d-9064-c471df75d41a admin admin] PATH_INFO: `/v3/projects/b6a00e39073343df9a8a0ceffb9c13d2/users/9e80f97f50af43cc92fc36fc57aae3bd/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:57.660250 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816e32b-6885-426b-8c5d-1be76fbe4e5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:57.660250 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816e32b-6885-426b-8c5d-1be76fbe4e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:57.660250 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816e32b-6885-426b-8c5d-1be76fbe4e5e admin admin] PATH_INFO: `/v3/projects/a61adf5ef5c94189806eede6cf2b6bc5/users/ab0f5064f76d43128861271efc29db54/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:57.676940 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816e32b-6885-426b-8c5d-1be76fbe4e5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a61adf5ef5c94189806eede6cf2b6bc5, user_id=ab0f5064f76d43128861271efc29db54, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:57.678661 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06a629-691c-437d-9064-c471df75d41a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6a00e39073343df9a8a0ceffb9c13d2, user_id=9e80f97f50af43cc92fc36fc57aae3bd, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:57.679865 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816e32b-6885-426b-8c5d-1be76fbe4e5e admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:57.687299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06a629-691c-437d-9064-c471df75d41a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:57.696918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 981/1960] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:57 2022] PUT /identity/v3/projects/a61adf5ef5c94189806eede6cf2b6bc5/users/ab0f5064f76d43128861271efc29db54/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:38:57.708076 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f98b42-b494-4fb8-aeb8-c0070aee992e None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:57.708403 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f98b42-b494-4fb8-aeb8-c0070aee992e None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:57.708656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f98b42-b494-4fb8-aeb8-c0070aee992e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:57.708828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 980/1961] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:57 2022] PUT /identity/v3/projects/b6a00e39073343df9a8a0ceffb9c13d2/users/9e80f97f50af43cc92fc36fc57aae3bd/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:38:57.718839 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213df20e-40fc-4623-822d-09aa670baef7 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:57.719192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213df20e-40fc-4623-822d-09aa670baef7 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:57.719721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213df20e-40fc-4623-822d-09aa670baef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:57.737714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-a9f98b42-b494-4fb8-aeb8-c0070aee992e None None] MFA Rules not processed for user `ab0f5064f76d43128861271efc29db54`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:57.738941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a9f98b42-b494-4fb8-aeb8-c0070aee992e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:57.750009 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-213df20e-40fc-4623-822d-09aa670baef7 None None] MFA Rules not processed for user `9e80f97f50af43cc92fc36fc57aae3bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:57.751720 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-213df20e-40fc-4623-822d-09aa670baef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:57.819569 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 982/1962] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:38:57.828439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f98b42-b494-4fb8-aeb8-c0070aee992e None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:57.830253 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a9f98b42-b494-4fb8-aeb8-c0070aee992e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:57.831492 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 981/1963] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:57.835332 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213df20e-40fc-4623-822d-09aa670baef7 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:57.851981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:57.902431 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:57.902699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f98b42-b494-4fb8-aeb8-c0070aee992e None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:57.907098 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03739287-cfda-4d4a-b458-f3ab6df284a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:57.907875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03739287-cfda-4d4a-b458-f3ab6df284a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:57.908100 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03739287-cfda-4d4a-b458-f3ab6df284a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:57.908371 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03739287-cfda-4d4a-b458-f3ab6df284a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:57.920701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03739287-cfda-4d4a-b458-f3ab6df284a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:57.922141 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03739287-cfda-4d4a-b458-f3ab6df284a5 service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:57.924956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213df20e-40fc-4623-822d-09aa670baef7 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:57.934920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 983/1964] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:38:57.935178 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647a465-edee-4e35-8229-6c5c02f21a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:57.935472 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647a465-edee-4e35-8229-6c5c02f21a5d admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:57.935922 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647a465-edee-4e35-8229-6c5c02f21a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:57.936206 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647a465-edee-4e35-8229-6c5c02f21a5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:57.936917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647a465-edee-4e35-8229-6c5c02f21a5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:57.939690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647a465-edee-4e35-8229-6c5c02f21a5d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:57.955321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03739287-cfda-4d4a-b458-f3ab6df284a5 service cinder] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:57.973636 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03739287-cfda-4d4a-b458-f3ab6df284a5 service cinder] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:57.978802 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a290c-973b-4f5b-a64e-4ad4b09ad532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:57.979685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a290c-973b-4f5b-a64e-4ad4b09ad532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:57.980004 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a290c-973b-4f5b-a64e-4ad4b09ad532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:57.980278 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a290c-973b-4f5b-a64e-4ad4b09ad532 admin admin] PATH_INFO: `/v3/users/ab0f5064f76d43128861271efc29db54` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:57.984331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 982/1965] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:57 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:38:57.984649 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a290c-973b-4f5b-a64e-4ad4b09ad532 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab0f5064f76d43128861271efc29db54)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:57.985269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647a465-edee-4e35-8229-6c5c02f21a5d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:57.986588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a290c-973b-4f5b-a64e-4ad4b09ad532 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:58.086937 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-4d9a290c-973b-4f5b-a64e-4ad4b09ad532 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab0f5064f76d43128861271efc29db54'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:58.088403 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647a465-edee-4e35-8229-6c5c02f21a5d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:58.094415 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc6505-285d-4973-9626-208a7596d3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:58.095103 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc6505-285d-4973-9626-208a7596d3a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:58.095942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc6505-285d-4973-9626-208a7596d3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:58.095942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc6505-285d-4973-9626-208a7596d3a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:58.098962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc6505-285d-4973-9626-208a7596d3a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:58.098962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc6505-285d-4973-9626-208a7596d3a3 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:58.103218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-4d9a290c-973b-4f5b-a64e-4ad4b09ad532 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab0f5064f76d43128861271efc29db54'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:58.116482 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-4d9a290c-973b-4f5b-a64e-4ad4b09ad532 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab0f5064f76d43128861271efc29db54'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:58.117325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-4d9a290c-973b-4f5b-a64e-4ad4b09ad532 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab0f5064f76d43128861271efc29db54'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:58.127857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 983/1966] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:57 2022] POST /identity/v3/users => generated 298 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:58.131285 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc6505-285d-4973-9626-208a7596d3a3 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:58.138995 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-4d9a290c-973b-4f5b-a64e-4ad4b09ad532 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab0f5064f76d43128861271efc29db54'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:58.142697 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc6505-285d-4973-9626-208a7596d3a3 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:58.146590 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 984/1967] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:38:57 2022] DELETE /identity/v3/users/ab0f5064f76d43128861271efc29db54 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:58.148186 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52616f7f-731f-45cf-b045-57fc1cdbc0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:58.148855 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52616f7f-731f-45cf-b045-57fc1cdbc0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:58.149135 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52616f7f-731f-45cf-b045-57fc1cdbc0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:58.149506 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52616f7f-731f-45cf-b045-57fc1cdbc0b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:58.150247 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52616f7f-731f-45cf-b045-57fc1cdbc0b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:58.151113 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707adce-6afd-4333-abf3-3a5a5e466d6a None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:58.151511 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707adce-6afd-4333-abf3-3a5a5e466d6a None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:58.151867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52616f7f-731f-45cf-b045-57fc1cdbc0b4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:58.151976 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707adce-6afd-4333-abf3-3a5a5e466d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:58.167684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 984/1968] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:38:58.168933 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52616f7f-731f-45cf-b045-57fc1cdbc0b4 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:58.170994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-52616f7f-731f-45cf-b045-57fc1cdbc0b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:58.172091 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-0707adce-6afd-4333-abf3-3a5a5e466d6a None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:58.173425 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-0707adce-6afd-4333-abf3-3a5a5e466d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:58.187070 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52616f7f-731f-45cf-b045-57fc1cdbc0b4 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:58.247633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 985/1969] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:58.250272 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25df179-4c21-4aa4-9f16-de7378196c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:58.251379 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25df179-4c21-4aa4-9f16-de7378196c19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:58.251913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25df179-4c21-4aa4-9f16-de7378196c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:58.252293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25df179-4c21-4aa4-9f16-de7378196c19 admin admin] PATH_INFO: `/v3/projects/111c20f9e83941e8aa4baecd6b3369e0/users/4dcf997b051b4003b70a477d419ad9d6/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:58.259747 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707adce-6afd-4333-abf3-3a5a5e466d6a None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:58.261171 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-0707adce-6afd-4333-abf3-3a5a5e466d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:58.269332 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25df179-4c21-4aa4-9f16-de7378196c19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=111c20f9e83941e8aa4baecd6b3369e0, user_id=4dcf997b051b4003b70a477d419ad9d6, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:58.272496 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25df179-4c21-4aa4-9f16-de7378196c19 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:58.298488 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 985/1970] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:58 2022] PUT /identity/v3/projects/111c20f9e83941e8aa4baecd6b3369e0/users/4dcf997b051b4003b70a477d419ad9d6/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:58.300338 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25df179-4c21-4aa4-9f16-de7378196c19 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:58.313170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:58.354826 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:58.361453 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707adce-6afd-4333-abf3-3a5a5e466d6a None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:58.413998 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25df179-4c21-4aa4-9f16-de7378196c19 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:58.419708 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e415eb3-f927-4495-9d6b-c65e798cf945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:58.420607 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e415eb3-f927-4495-9d6b-c65e798cf945 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:58.420896 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e415eb3-f927-4495-9d6b-c65e798cf945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:58.421519 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e415eb3-f927-4495-9d6b-c65e798cf945 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:58.422261 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e415eb3-f927-4495-9d6b-c65e798cf945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:58.423899 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e415eb3-f927-4495-9d6b-c65e798cf945 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:58.434818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4e238-2b9f-457f-92f5-0d763414d4c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:58.435392 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 986/1971] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:58.435506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4e238-2b9f-457f-92f5-0d763414d4c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:58.435743 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4e238-2b9f-457f-92f5-0d763414d4c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:58.436050 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4e238-2b9f-457f-92f5-0d763414d4c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:58.439889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e415eb3-f927-4495-9d6b-c65e798cf945 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:58.451251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4e238-2b9f-457f-92f5-0d763414d4c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:58.453043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4e238-2b9f-457f-92f5-0d763414d4c5 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:58.454780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e415eb3-f927-4495-9d6b-c65e798cf945 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:58.460690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23b308-ea4d-4a36-ad75-387b6d3f5b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:58.461922 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23b308-ea4d-4a36-ad75-387b6d3f5b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:58.462208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23b308-ea4d-4a36-ad75-387b6d3f5b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:58.462500 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23b308-ea4d-4a36-ad75-387b6d3f5b43 admin admin] PATH_INFO: `/v3/projects/111c20f9e83941e8aa4baecd6b3369e0/users/4dcf997b051b4003b70a477d419ad9d6/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:58.464814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23b308-ea4d-4a36-ad75-387b6d3f5b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=111c20f9e83941e8aa4baecd6b3369e0, user_id=4dcf997b051b4003b70a477d419ad9d6, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:58.468664 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23b308-ea4d-4a36-ad75-387b6d3f5b43 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:58.470224 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 986/1972] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:38:58.499876 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 987/1973] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:58 2022] PUT /identity/v3/projects/111c20f9e83941e8aa4baecd6b3369e0/users/4dcf997b051b4003b70a477d419ad9d6/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:58.504868 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4524a85-70ae-4254-82f9-746fd67d9a6f None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:58.505376 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4524a85-70ae-4254-82f9-746fd67d9a6f None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:58.505376 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4524a85-70ae-4254-82f9-746fd67d9a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:58.537405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-a4524a85-70ae-4254-82f9-746fd67d9a6f None None] MFA Rules not processed for user `4dcf997b051b4003b70a477d419ad9d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:58.538618 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a4524a85-70ae-4254-82f9-746fd67d9a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:58.642446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 987/1974] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:58.662023 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23b308-ea4d-4a36-ad75-387b6d3f5b43 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:58.664889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2d23b308-ea4d-4a36-ad75-387b6d3f5b43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:58.740854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23b308-ea4d-4a36-ad75-387b6d3f5b43 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:58.745745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c4359-d5a5-4b66-ab98-da4980a6e20a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:58.746606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c4359-d5a5-4b66-ab98-da4980a6e20a service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:58.747096 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c4359-d5a5-4b66-ab98-da4980a6e20a service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:58.747538 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c4359-d5a5-4b66-ab98-da4980a6e20a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:58.749700 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bd8c4359-d5a5-4b66-ab98-da4980a6e20a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:58.836233 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4524a85-70ae-4254-82f9-746fd67d9a6f None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:58.839198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a4524a85-70ae-4254-82f9-746fd67d9a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:58.849047 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c4359-d5a5-4b66-ab98-da4980a6e20a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:58.850521 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c4359-d5a5-4b66-ab98-da4980a6e20a service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:58.867693 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 988/1975] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:38:58.944366 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4524a85-70ae-4254-82f9-746fd67d9a6f None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:58.950216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f12c3-7b89-437b-9346-b61450142aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:58.951377 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f12c3-7b89-437b-9346-b61450142aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:58.951802 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f12c3-7b89-437b-9346-b61450142aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:58.952184 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f12c3-7b89-437b-9346-b61450142aba admin admin] PATH_INFO: `/v3/projects/a61adf5ef5c94189806eede6cf2b6bc5` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:58.953536 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f12c3-7b89-437b-9346-b61450142aba admin admin] RBAC: Authorizing `identity:delete_project(project_id=a61adf5ef5c94189806eede6cf2b6bc5)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:58.955656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f12c3-7b89-437b-9346-b61450142aba admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:58.974630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:59.021214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3a6f12c3-7b89-437b-9346-b61450142aba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a61adf5ef5c94189806eede6cf2b6bc5', 'request_id': 'req-3a6f12c3-7b89-437b-9346-b61450142aba'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:59.030888 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3a6f12c3-7b89-437b-9346-b61450142aba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a61adf5ef5c94189806eede6cf2b6bc5', 'request_id': 'req-3a6f12c3-7b89-437b-9346-b61450142aba'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:59.039856 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3a6f12c3-7b89-437b-9346-b61450142aba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a61adf5ef5c94189806eede6cf2b6bc5', 'request_id': 'req-3a6f12c3-7b89-437b-9346-b61450142aba'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:59.052997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3a6f12c3-7b89-437b-9346-b61450142aba admin admin] The token cache is being invalidate because project a61adf5ef5c94189806eede6cf2b6bc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:38:59.053419 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3a6f12c3-7b89-437b-9346-b61450142aba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:59.054218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3a6f12c3-7b89-437b-9346-b61450142aba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:38:59.056429 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 988/1976] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:38:58 2022] DELETE /identity/v3/projects/a61adf5ef5c94189806eede6cf2b6bc5 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:59.062401 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b474f-0fd7-492d-b072-553d02df6b1e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:59.062759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b474f-0fd7-492d-b072-553d02df6b1e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:59.063082 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b474f-0fd7-492d-b072-553d02df6b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:59.094181 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-fe8b474f-0fd7-492d-b072-553d02df6b1e None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:59.095695 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fe8b474f-0fd7-492d-b072-553d02df6b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:59.210544 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb115cb-783d-4f09-bc91-ef79d76cde1a None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:59.210834 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb115cb-783d-4f09-bc91-ef79d76cde1a None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:59.211639 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb115cb-783d-4f09-bc91-ef79d76cde1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:59.219574 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 989/1977] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:38:59.245508 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-bfb115cb-783d-4f09-bc91-ef79d76cde1a None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:59.248100 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-bfb115cb-783d-4f09-bc91-ef79d76cde1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:59.258103 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 989/1978] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:38:59.263468 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b474f-0fd7-492d-b072-553d02df6b1e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:59.279510 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b474f-0fd7-492d-b072-553d02df6b1e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:59.286577 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d52c96-4e34-403e-b20d-7b21e2d27d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:59.287388 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d52c96-4e34-403e-b20d-7b21e2d27d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:59.287857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d52c96-4e34-403e-b20d-7b21e2d27d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:59.288147 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d52c96-4e34-403e-b20d-7b21e2d27d4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:59.288887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d52c96-4e34-403e-b20d-7b21e2d27d4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:38:59.289198 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d52c96-4e34-403e-b20d-7b21e2d27d4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:59.294312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d52c96-4e34-403e-b20d-7b21e2d27d4d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:59.307747 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:38:59.317138 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb115cb-783d-4f09-bc91-ef79d76cde1a None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:59.320584 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 990/1979] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:38:59.328043 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb115cb-783d-4f09-bc91-ef79d76cde1a None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:59.339060 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768296e-493d-4445-a9f2-9dbde18b7169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:59.339921 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768296e-493d-4445-a9f2-9dbde18b7169 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:59.340294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768296e-493d-4445-a9f2-9dbde18b7169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:59.341428 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768296e-493d-4445-a9f2-9dbde18b7169 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:59.342727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b768296e-493d-4445-a9f2-9dbde18b7169 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:59.348916 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b768296e-493d-4445-a9f2-9dbde18b7169 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:59.378680 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 990/1980] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:38:59 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:38:59.382894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d52c96-4e34-403e-b20d-7b21e2d27d4d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:59.395694 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d52c96-4e34-403e-b20d-7b21e2d27d4d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:59.458353 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e00780c-930e-4152-be84-ce65ada2d8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:59.459038 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00780c-930e-4152-be84-ce65ada2d8f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:59.459317 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00780c-930e-4152-be84-ce65ada2d8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:59.459585 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00780c-930e-4152-be84-ce65ada2d8f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:59.460310 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e00780c-930e-4152-be84-ce65ada2d8f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:59.462233 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e00780c-930e-4152-be84-ce65ada2d8f0 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:59.489044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 991/1981] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:38:59 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:38:59.492938 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768296e-493d-4445-a9f2-9dbde18b7169 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:59.504839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768296e-493d-4445-a9f2-9dbde18b7169 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:59.510724 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb6d6c-cff8-4d0f-a047-1209295de18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:59.511405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb6d6c-cff8-4d0f-a047-1209295de18f admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:59.511688 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb6d6c-cff8-4d0f-a047-1209295de18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:59.512091 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb6d6c-cff8-4d0f-a047-1209295de18f admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:59.513209 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb6d6c-cff8-4d0f-a047-1209295de18f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:59.515275 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb6d6c-cff8-4d0f-a047-1209295de18f admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:59.530335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 991/1982] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:59.536947 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e00780c-930e-4152-be84-ce65ada2d8f0 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:59.547829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e00780c-930e-4152-be84-ce65ada2d8f0 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:59.553080 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de4184-b15e-4941-9890-06327581dd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:59.553534 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de4184-b15e-4941-9890-06327581dd83 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:59.553678 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de4184-b15e-4941-9890-06327581dd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:59.553959 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de4184-b15e-4941-9890-06327581dd83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:59.554938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de4184-b15e-4941-9890-06327581dd83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:59.556611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de4184-b15e-4941-9890-06327581dd83 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:59.568032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 992/1983] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:38:59.572123 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb6d6c-cff8-4d0f-a047-1209295de18f admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:59.582720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb6d6c-cff8-4d0f-a047-1209295de18f admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:59.587315 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acae150-6d9b-4486-a6db-ee72c9937ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:59.588249 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acae150-6d9b-4486-a6db-ee72c9937ab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:59.588530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acae150-6d9b-4486-a6db-ee72c9937ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:59.588803 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acae150-6d9b-4486-a6db-ee72c9937ab9 admin admin] PATH_INFO: `/v3/projects/147325fac7434eabbdb23fef3c25199d/users/815d456fbb9b4bd5b72fbb7d6249dadd/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:59.603471 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acae150-6d9b-4486-a6db-ee72c9937ab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=147325fac7434eabbdb23fef3c25199d, user_id=815d456fbb9b4bd5b72fbb7d6249dadd, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:59.606204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acae150-6d9b-4486-a6db-ee72c9937ab9 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:59.620062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 992/1984] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:38:59 2022] PUT /identity/v3/projects/147325fac7434eabbdb23fef3c25199d/users/815d456fbb9b4bd5b72fbb7d6249dadd/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:38:59.631570 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b104cb3-c4b0-49c0-b883-94a3991f3991 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:59.631893 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b104cb3-c4b0-49c0-b883-94a3991f3991 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:59.632174 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b104cb3-c4b0-49c0-b883-94a3991f3991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:59.659732 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2b104cb3-c4b0-49c0-b883-94a3991f3991 None None] MFA Rules not processed for user `815d456fbb9b4bd5b72fbb7d6249dadd`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:38:59.661129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2b104cb3-c4b0-49c0-b883-94a3991f3991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:59.731937 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 993/1985] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:38:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:38:59.741988 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acae150-6d9b-4486-a6db-ee72c9937ab9 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:38:59.743688 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-8acae150-6d9b-4486-a6db-ee72c9937ab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:38:59.801147 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acae150-6d9b-4486-a6db-ee72c9937ab9 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:38:59.804724 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecf422-b988-4959-a91a-dce9a8c06eaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:38:59.805356 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecf422-b988-4959-a91a-dce9a8c06eaf service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:38:59.805644 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecf422-b988-4959-a91a-dce9a8c06eaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:38:59.805921 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecf422-b988-4959-a91a-dce9a8c06eaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:38:59.814801 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eecf422-b988-4959-a91a-dce9a8c06eaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:38:59.816261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eecf422-b988-4959-a91a-dce9a8c06eaf service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:38:59.825720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 993/1986] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:38:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:39:01.521924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b104cb3-c4b0-49c0-b883-94a3991f3991 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:01.531760 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b104cb3-c4b0-49c0-b883-94a3991f3991 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:01.535303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a0158-1b38-480a-a63d-27ce0b6cf291 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:01.535979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a0158-1b38-480a-a63d-27ce0b6cf291 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:01.536263 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a0158-1b38-480a-a63d-27ce0b6cf291 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:01.536540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a0158-1b38-480a-a63d-27ce0b6cf291 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:01.538308 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-400a0158-1b38-480a-a63d-27ce0b6cf291 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:01.598099 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a0158-1b38-480a-a63d-27ce0b6cf291 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:01.600139 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a0158-1b38-480a-a63d-27ce0b6cf291 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:01.610149 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 994/1987] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:39:03.306857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecf422-b988-4959-a91a-dce9a8c06eaf service cinder] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:03.308357 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-5eecf422-b988-4959-a91a-dce9a8c06eaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:03.320343 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecf422-b988-4959-a91a-dce9a8c06eaf service cinder] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:03.324296 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e303fae-dc15-43c5-b7dd-dfc709bceb27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:03.325665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e303fae-dc15-43c5-b7dd-dfc709bceb27 service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:03.326149 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e303fae-dc15-43c5-b7dd-dfc709bceb27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:03.326577 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e303fae-dc15-43c5-b7dd-dfc709bceb27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:03.328774 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-0e303fae-dc15-43c5-b7dd-dfc709bceb27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:03.384346 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:03.412681 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e303fae-dc15-43c5-b7dd-dfc709bceb27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:03.414109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e303fae-dc15-43c5-b7dd-dfc709bceb27 service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:03.423549 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 994/1988] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:39:11.605389 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=94,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:11.868743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a0158-1b38-480a-a63d-27ce0b6cf291 service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:11.870278 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-400a0158-1b38-480a-a63d-27ce0b6cf291 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:11.891718 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e303fae-dc15-43c5-b7dd-dfc709bceb27 service cinder] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:11.964639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a0158-1b38-480a-a63d-27ce0b6cf291 service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:11.970770 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430be39-a4b0-4a0e-b640-ad66bf7ab2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:11.972027 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430be39-a4b0-4a0e-b640-ad66bf7ab2b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:11.972452 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430be39-a4b0-4a0e-b640-ad66bf7ab2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:11.972866 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430be39-a4b0-4a0e-b640-ad66bf7ab2b0 admin admin] PATH_INFO: `/v3/users/48aeafa9123a43329cfe85ef74dd62f9` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:11.974428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430be39-a4b0-4a0e-b640-ad66bf7ab2b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48aeafa9123a43329cfe85ef74dd62f9)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:11.976665 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430be39-a4b0-4a0e-b640-ad66bf7ab2b0 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:11.987991 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e303fae-dc15-43c5-b7dd-dfc709bceb27 service cinder] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:11.992889 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687725c5-1d1b-4b0a-8480-5524208e215d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:11.994113 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687725c5-1d1b-4b0a-8480-5524208e215d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:11.994582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687725c5-1d1b-4b0a-8480-5524208e215d admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:11.995072 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687725c5-1d1b-4b0a-8480-5524208e215d admin admin] PATH_INFO: `/v3/users/815d456fbb9b4bd5b72fbb7d6249dadd` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:11.997322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687725c5-1d1b-4b0a-8480-5524208e215d admin admin] RBAC: Authorizing `identity:delete_user(user_id=815d456fbb9b4bd5b72fbb7d6249dadd)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:11.999604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687725c5-1d1b-4b0a-8480-5524208e215d admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:12.055155 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d430be39-a4b0-4a0e-b640-ad66bf7ab2b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48aeafa9123a43329cfe85ef74dd62f9'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:12.059405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-687725c5-1d1b-4b0a-8480-5524208e215d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '815d456fbb9b4bd5b72fbb7d6249dadd'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:12.065362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d430be39-a4b0-4a0e-b640-ad66bf7ab2b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48aeafa9123a43329cfe85ef74dd62f9'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:12.066194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d430be39-a4b0-4a0e-b640-ad66bf7ab2b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48aeafa9123a43329cfe85ef74dd62f9'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:12.067059 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d430be39-a4b0-4a0e-b640-ad66bf7ab2b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48aeafa9123a43329cfe85ef74dd62f9'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:12.068787 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-687725c5-1d1b-4b0a-8480-5524208e215d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '815d456fbb9b4bd5b72fbb7d6249dadd'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:12.078733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-687725c5-1d1b-4b0a-8480-5524208e215d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '815d456fbb9b4bd5b72fbb7d6249dadd'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:12.079510 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-687725c5-1d1b-4b0a-8480-5524208e215d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '815d456fbb9b4bd5b72fbb7d6249dadd'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:12.093282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d430be39-a4b0-4a0e-b640-ad66bf7ab2b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48aeafa9123a43329cfe85ef74dd62f9'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:12.098687 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-687725c5-1d1b-4b0a-8480-5524208e215d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '815d456fbb9b4bd5b72fbb7d6249dadd'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:12.107024 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 995/1989] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:39:11 2022] DELETE /identity/v3/users/815d456fbb9b4bd5b72fbb7d6249dadd => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:39:12.111244 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb52416-0f40-4458-bb99-362e5fa7b3fe None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:12.111679 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb52416-0f40-4458-bb99-362e5fa7b3fe None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:12.115310 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 995/1990] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:39:11 2022] DELETE /identity/v3/users/48aeafa9123a43329cfe85ef74dd62f9 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:12.115633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb52416-0f40-4458-bb99-362e5fa7b3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:12.117705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209892c-3419-4e39-9146-68287c492fd7 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:12.118604 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209892c-3419-4e39-9146-68287c492fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:12.119351 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209892c-3419-4e39-9146-68287c492fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:12.133980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-afb52416-0f40-4458-bb99-362e5fa7b3fe None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:12.133980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-afb52416-0f40-4458-bb99-362e5fa7b3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:12.145827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3209892c-3419-4e39-9146-68287c492fd7 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:12.148630 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3209892c-3419-4e39-9146-68287c492fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:12.214577 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:12.234140 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 996/1991] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:39:12.235208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 996/1992] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:39:12.248723 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb52416-0f40-4458-bb99-362e5fa7b3fe None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:12.248888 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209892c-3419-4e39-9146-68287c492fd7 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:12.250137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-afb52416-0f40-4458-bb99-362e5fa7b3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:12.250247 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3209892c-3419-4e39-9146-68287c492fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:12.275959 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:12.372616 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb52416-0f40-4458-bb99-362e5fa7b3fe None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:12.377994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d5ca8-7ef8-4643-b90a-fb2a8d7226f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:12.379177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d5ca8-7ef8-4643-b90a-fb2a8d7226f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:12.379471 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d5ca8-7ef8-4643-b90a-fb2a8d7226f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:12.381441 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d5ca8-7ef8-4643-b90a-fb2a8d7226f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:12.396593 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d5ca8-7ef8-4643-b90a-fb2a8d7226f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:12.399549 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d5ca8-7ef8-4643-b90a-fb2a8d7226f2 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:12.424624 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209892c-3419-4e39-9146-68287c492fd7 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:12.433872 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 997/1993] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:39:12.434875 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5c743-b9cd-4d08-b39c-63fab129cf65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:12.435613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5c743-b9cd-4d08-b39c-63fab129cf65 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:12.435951 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5c743-b9cd-4d08-b39c-63fab129cf65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:12.436567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5c743-b9cd-4d08-b39c-63fab129cf65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:12.450757 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5c743-b9cd-4d08-b39c-63fab129cf65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:12.453305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5c743-b9cd-4d08-b39c-63fab129cf65 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:12.466821 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 997/1994] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:39:12.894072 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d5ca8-7ef8-4643-b90a-fb2a8d7226f2 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:12.896398 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-070d5ca8-7ef8-4643-b90a-fb2a8d7226f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:12.911058 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d5ca8-7ef8-4643-b90a-fb2a8d7226f2 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:12.926538 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0895b2-ca84-43ff-b579-1fb1bc4d90a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:12.927368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0895b2-ca84-43ff-b579-1fb1bc4d90a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:12.927646 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0895b2-ca84-43ff-b579-1fb1bc4d90a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:12.928397 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0895b2-ca84-43ff-b579-1fb1bc4d90a3 admin admin] PATH_INFO: `/v3/projects/147325fac7434eabbdb23fef3c25199d` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:12.929803 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0895b2-ca84-43ff-b579-1fb1bc4d90a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=147325fac7434eabbdb23fef3c25199d)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:12.931582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0895b2-ca84-43ff-b579-1fb1bc4d90a3 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:12.963603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5c743-b9cd-4d08-b39c-63fab129cf65 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:12.965347 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4ba5c743-b9cd-4d08-b39c-63fab129cf65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:12.979678 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5c743-b9cd-4d08-b39c-63fab129cf65 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:12.990785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7e0895b2-ca84-43ff-b579-1fb1bc4d90a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '147325fac7434eabbdb23fef3c25199d', 'request_id': 'req-7e0895b2-ca84-43ff-b579-1fb1bc4d90a3'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:12.997660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7e0895b2-ca84-43ff-b579-1fb1bc4d90a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '147325fac7434eabbdb23fef3c25199d', 'request_id': 'req-7e0895b2-ca84-43ff-b579-1fb1bc4d90a3'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.010807 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7e0895b2-ca84-43ff-b579-1fb1bc4d90a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '147325fac7434eabbdb23fef3c25199d', 'request_id': 'req-7e0895b2-ca84-43ff-b579-1fb1bc4d90a3'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.028176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7e0895b2-ca84-43ff-b579-1fb1bc4d90a3 admin admin] The token cache is being invalidate because project 147325fac7434eabbdb23fef3c25199d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:39:13.028615 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7e0895b2-ca84-43ff-b579-1fb1bc4d90a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.029550 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7e0895b2-ca84-43ff-b579-1fb1bc4d90a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.032260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 998/1995] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:39:12 2022] DELETE /identity/v3/projects/147325fac7434eabbdb23fef3c25199d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:13.039320 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ccca7-8e35-4dd9-b6cb-8639b534d1d5 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:13.039880 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ccca7-8e35-4dd9-b6cb-8639b534d1d5 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:13.040908 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ccca7-8e35-4dd9-b6cb-8639b534d1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:13.058503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedab66f-a794-45a8-91a3-22c6d61f6b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:13.061547 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedab66f-a794-45a8-91a3-22c6d61f6b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:13.061844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedab66f-a794-45a8-91a3-22c6d61f6b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:13.062195 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedab66f-a794-45a8-91a3-22c6d61f6b66 admin admin] PATH_INFO: `/v3/users/3bbc3cd193444a65861f6e6705fd32cf` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:13.065721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedab66f-a794-45a8-91a3-22c6d61f6b66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bbc3cd193444a65861f6e6705fd32cf)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:13.065721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedab66f-a794-45a8-91a3-22c6d61f6b66 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:13.073321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-bf6ccca7-8e35-4dd9-b6cb-8639b534d1d5 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:13.075215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-bf6ccca7-8e35-4dd9-b6cb-8639b534d1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:13.095692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 999/1996] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:39:13.095692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ccca7-8e35-4dd9-b6cb-8639b534d1d5 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:13.095692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-bf6ccca7-8e35-4dd9-b6cb-8639b534d1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:13.109602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ccca7-8e35-4dd9-b6cb-8639b534d1d5 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:13.115700 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305f579-388d-4bdc-94ef-2914be67172a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:13.116413 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305f579-388d-4bdc-94ef-2914be67172a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:13.116705 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305f579-388d-4bdc-94ef-2914be67172a admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:13.116986 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305f579-388d-4bdc-94ef-2914be67172a admin admin] PATH_INFO: `/v3/projects/cb6299abef70443b9a5d522f19b5fb5e` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:13.118185 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305f579-388d-4bdc-94ef-2914be67172a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb6299abef70443b9a5d522f19b5fb5e)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:13.120439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305f579-388d-4bdc-94ef-2914be67172a admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:13.131389 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bedab66f-a794-45a8-91a3-22c6d61f6b66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bbc3cd193444a65861f6e6705fd32cf'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.139504 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bedab66f-a794-45a8-91a3-22c6d61f6b66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bbc3cd193444a65861f6e6705fd32cf'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.140287 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bedab66f-a794-45a8-91a3-22c6d61f6b66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bbc3cd193444a65861f6e6705fd32cf'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.140904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bedab66f-a794-45a8-91a3-22c6d61f6b66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bbc3cd193444a65861f6e6705fd32cf'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.153991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bedab66f-a794-45a8-91a3-22c6d61f6b66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bbc3cd193444a65861f6e6705fd32cf'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.173051 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 998/1997] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:39:12 2022] DELETE /identity/v3/users/3bbc3cd193444a65861f6e6705fd32cf => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:39:13.173051 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede9790-41d7-4020-bcc9-1a8e5536e02b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:13.173051 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede9790-41d7-4020-bcc9-1a8e5536e02b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:13.173051 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede9790-41d7-4020-bcc9-1a8e5536e02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:13.190008 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1305f579-388d-4bdc-94ef-2914be67172a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb6299abef70443b9a5d522f19b5fb5e', 'request_id': 'req-1305f579-388d-4bdc-94ef-2914be67172a'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.199015 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-1ede9790-41d7-4020-bcc9-1a8e5536e02b None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:13.201181 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1ede9790-41d7-4020-bcc9-1a8e5536e02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:13.202799 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1305f579-388d-4bdc-94ef-2914be67172a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb6299abef70443b9a5d522f19b5fb5e', 'request_id': 'req-1305f579-388d-4bdc-94ef-2914be67172a'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.211634 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1305f579-388d-4bdc-94ef-2914be67172a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb6299abef70443b9a5d522f19b5fb5e', 'request_id': 'req-1305f579-388d-4bdc-94ef-2914be67172a'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.222204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:13.227758 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1305f579-388d-4bdc-94ef-2914be67172a admin admin] The token cache is being invalidate because project cb6299abef70443b9a5d522f19b5fb5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:39:13.228280 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1305f579-388d-4bdc-94ef-2914be67172a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.229893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1305f579-388d-4bdc-94ef-2914be67172a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.235825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1000/1998] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:39:13 2022] DELETE /identity/v3/projects/cb6299abef70443b9a5d522f19b5fb5e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:13.236989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305f579-388d-4bdc-94ef-2914be67172a admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:13.238734 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-1305f579-388d-4bdc-94ef-2914be67172a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:13.264671 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:13.305515 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 999/1999] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:13.310178 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede9790-41d7-4020-bcc9-1a8e5536e02b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:13.319436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede9790-41d7-4020-bcc9-1a8e5536e02b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:13.325687 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305f579-388d-4bdc-94ef-2914be67172a admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:13.328227 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389c61c-2902-4410-9a3a-a8972fee34fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:13.328965 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389c61c-2902-4410-9a3a-a8972fee34fc admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:13.329188 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389c61c-2902-4410-9a3a-a8972fee34fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:13.329464 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389c61c-2902-4410-9a3a-a8972fee34fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:13.330203 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389c61c-2902-4410-9a3a-a8972fee34fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:39:13.330510 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389c61c-2902-4410-9a3a-a8972fee34fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:13.330622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e90d07-b620-4836-a6db-acad25ae87ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:13.331514 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e90d07-b620-4836-a6db-acad25ae87ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:13.331813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e90d07-b620-4836-a6db-acad25ae87ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:13.332177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e90d07-b620-4836-a6db-acad25ae87ad admin admin] PATH_INFO: `/v3/users/9e80f97f50af43cc92fc36fc57aae3bd` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:13.332347 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389c61c-2902-4410-9a3a-a8972fee34fc admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:13.333418 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e90d07-b620-4836-a6db-acad25ae87ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e80f97f50af43cc92fc36fc57aae3bd)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:13.335332 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e90d07-b620-4836-a6db-acad25ae87ad admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:13.344766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1000/2000] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:39:13.349846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389c61c-2902-4410-9a3a-a8972fee34fc admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:13.363948 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389c61c-2902-4410-9a3a-a8972fee34fc admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:13.370599 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96535c70-05c3-437b-897a-c27392310c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:13.372727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96535c70-05c3-437b-897a-c27392310c34 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:13.373020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96535c70-05c3-437b-897a-c27392310c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:13.373302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96535c70-05c3-437b-897a-c27392310c34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:13.374108 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96535c70-05c3-437b-897a-c27392310c34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:13.376768 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96535c70-05c3-437b-897a-c27392310c34 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:13.401426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e4e90d07-b620-4836-a6db-acad25ae87ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e80f97f50af43cc92fc36fc57aae3bd'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.412206 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1001/2001] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:39:13 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:13.413316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e4e90d07-b620-4836-a6db-acad25ae87ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e80f97f50af43cc92fc36fc57aae3bd'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.415923 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96535c70-05c3-437b-897a-c27392310c34 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:13.417592 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-96535c70-05c3-437b-897a-c27392310c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:13.425341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e4e90d07-b620-4836-a6db-acad25ae87ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e80f97f50af43cc92fc36fc57aae3bd'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.426997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e4e90d07-b620-4836-a6db-acad25ae87ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e80f97f50af43cc92fc36fc57aae3bd'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.448661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e4e90d07-b620-4836-a6db-acad25ae87ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e80f97f50af43cc92fc36fc57aae3bd'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.464402 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1001/2002] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:39:13 2022] DELETE /identity/v3/users/9e80f97f50af43cc92fc36fc57aae3bd => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:13.467663 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e90d07-b620-4836-a6db-acad25ae87ad admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:13.468197 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-e4e90d07-b620-4836-a6db-acad25ae87ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:13.500473 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96535c70-05c3-437b-897a-c27392310c34 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:13.585862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c27438-2bf2-4900-a748-9d9baabd421c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:13.586731 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c27438-2bf2-4900-a748-9d9baabd421c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:13.587323 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c27438-2bf2-4900-a748-9d9baabd421c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:13.587564 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c27438-2bf2-4900-a748-9d9baabd421c admin admin] PATH_INFO: `/v3/projects/8486d765afed40dc8795c7ba485d9535` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:13.589113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c27438-2bf2-4900-a748-9d9baabd421c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8486d765afed40dc8795c7ba485d9535)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:13.591737 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c27438-2bf2-4900-a748-9d9baabd421c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:13.595590 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e90d07-b620-4836-a6db-acad25ae87ad admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:13.601706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d52c53-cf88-4b1e-baee-c5ff6dd422b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:13.602376 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d52c53-cf88-4b1e-baee-c5ff6dd422b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:13.602664 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d52c53-cf88-4b1e-baee-c5ff6dd422b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:13.602940 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d52c53-cf88-4b1e-baee-c5ff6dd422b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:13.604193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d52c53-cf88-4b1e-baee-c5ff6dd422b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:13.607320 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d52c53-cf88-4b1e-baee-c5ff6dd422b1 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:13.633296 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1002/2003] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:39:13 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:13.637702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d52c53-cf88-4b1e-baee-c5ff6dd422b1 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:13.655164 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d52c53-cf88-4b1e-baee-c5ff6dd422b1 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:13.658194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-73c27438-2bf2-4900-a748-9d9baabd421c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8486d765afed40dc8795c7ba485d9535', 'request_id': 'req-73c27438-2bf2-4900-a748-9d9baabd421c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.669260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-73c27438-2bf2-4900-a748-9d9baabd421c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8486d765afed40dc8795c7ba485d9535', 'request_id': 'req-73c27438-2bf2-4900-a748-9d9baabd421c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.670809 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:13.675643 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:13.679145 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-73c27438-2bf2-4900-a748-9d9baabd421c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8486d765afed40dc8795c7ba485d9535', 'request_id': 'req-73c27438-2bf2-4900-a748-9d9baabd421c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.697105 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-73c27438-2bf2-4900-a748-9d9baabd421c admin admin] The token cache is being invalidate because project 8486d765afed40dc8795c7ba485d9535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:39:13.698303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-73c27438-2bf2-4900-a748-9d9baabd421c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.699287 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-73c27438-2bf2-4900-a748-9d9baabd421c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:13.701746 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1002/2004] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:39:13 2022] DELETE /identity/v3/projects/8486d765afed40dc8795c7ba485d9535 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:13.708316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d114a3-7437-4d8e-8c50-8ec60f25b33c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:13.708316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d114a3-7437-4d8e-8c50-8ec60f25b33c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:13.708835 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d114a3-7437-4d8e-8c50-8ec60f25b33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:13.729981 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-21d114a3-7437-4d8e-8c50-8ec60f25b33c None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:13.732325 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-21d114a3-7437-4d8e-8c50-8ec60f25b33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:13.749945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e675e-d76c-4c49-88df-f94ed3f1699b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:13.751207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e675e-d76c-4c49-88df-f94ed3f1699b admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:13.751553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e675e-d76c-4c49-88df-f94ed3f1699b admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:13.752224 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e675e-d76c-4c49-88df-f94ed3f1699b admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:13.752919 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e675e-d76c-4c49-88df-f94ed3f1699b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:13.754582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e675e-d76c-4c49-88df-f94ed3f1699b admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:13.769441 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1003/2005] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:13.774167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e675e-d76c-4c49-88df-f94ed3f1699b admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:13.776724 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-297e675e-d76c-4c49-88df-f94ed3f1699b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:13.792704 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e675e-d76c-4c49-88df-f94ed3f1699b admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:13.802989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5089b0-aa6e-43f0-93b0-0cfad0c7bce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:13.803679 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5089b0-aa6e-43f0-93b0-0cfad0c7bce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:13.804006 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5089b0-aa6e-43f0-93b0-0cfad0c7bce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:13.804285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5089b0-aa6e-43f0-93b0-0cfad0c7bce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:13.805019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5089b0-aa6e-43f0-93b0-0cfad0c7bce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:13.806620 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5089b0-aa6e-43f0-93b0-0cfad0c7bce1 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:13.830346 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1003/2007] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:13.832811 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1004/2006] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:39:13.833907 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d114a3-7437-4d8e-8c50-8ec60f25b33c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:13.840048 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9241b-22b9-4e24-9c2f-5378f7d5b57e None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:13.840577 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9241b-22b9-4e24-9c2f-5378f7d5b57e None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:13.840864 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9241b-22b9-4e24-9c2f-5378f7d5b57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:13.843978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d114a3-7437-4d8e-8c50-8ec60f25b33c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:13.851457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81cc08-aecf-4105-959a-52681a741bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:13.852415 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81cc08-aecf-4105-959a-52681a741bd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:13.852758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81cc08-aecf-4105-959a-52681a741bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:13.852956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81cc08-aecf-4105-959a-52681a741bd2 admin admin] PATH_INFO: `/v3/projects/b3b18ee23fb548709fc513c7e1cfe623/users/446b13b4661542c097afafb32443dae4/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:13.862327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-99c9241b-22b9-4e24-9c2f-5378f7d5b57e None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:13.863648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-99c9241b-22b9-4e24-9c2f-5378f7d5b57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:13.868727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81cc08-aecf-4105-959a-52681a741bd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3b18ee23fb548709fc513c7e1cfe623, user_id=446b13b4661542c097afafb32443dae4, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:13.871967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81cc08-aecf-4105-959a-52681a741bd2 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:13.874582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1005/2008] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:13.875560 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9241b-22b9-4e24-9c2f-5378f7d5b57e None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:13.877344 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-99c9241b-22b9-4e24-9c2f-5378f7d5b57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:13.886649 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1004/2009] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:13 2022] PUT /identity/v3/projects/b3b18ee23fb548709fc513c7e1cfe623/users/446b13b4661542c097afafb32443dae4/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:13.888498 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81cc08-aecf-4105-959a-52681a741bd2 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:13.891535 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9241b-22b9-4e24-9c2f-5378f7d5b57e None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:13.958289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1f548-66fc-49d5-9976-1378482dbdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:13.959189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1f548-66fc-49d5-9976-1378482dbdfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:13.959474 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1f548-66fc-49d5-9976-1378482dbdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:13.959565 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81cc08-aecf-4105-959a-52681a741bd2 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:13.959753 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1f548-66fc-49d5-9976-1378482dbdfb admin admin] PATH_INFO: `/v3/projects/b6a00e39073343df9a8a0ceffb9c13d2` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:13.960952 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1f548-66fc-49d5-9976-1378482dbdfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6a00e39073343df9a8a0ceffb9c13d2)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:13.962850 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1f548-66fc-49d5-9976-1378482dbdfb admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:13.964640 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28a7ee-b9e3-4e76-b71f-7d80495fc6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:13.965940 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28a7ee-b9e3-4e76-b71f-7d80495fc6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:13.966218 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28a7ee-b9e3-4e76-b71f-7d80495fc6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:13.966491 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28a7ee-b9e3-4e76-b71f-7d80495fc6d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:13.967215 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28a7ee-b9e3-4e76-b71f-7d80495fc6d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:39:13.967513 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28a7ee-b9e3-4e76-b71f-7d80495fc6d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:13.969131 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28a7ee-b9e3-4e76-b71f-7d80495fc6d2 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:13.980436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1005/2010] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:39:13.982135 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff85652-c234-49d2-bb2f-49a3b4d241bd None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:13.982421 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff85652-c234-49d2-bb2f-49a3b4d241bd None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:13.982696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff85652-c234-49d2-bb2f-49a3b4d241bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:14.001303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-fff85652-c234-49d2-bb2f-49a3b4d241bd None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:14.002494 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fff85652-c234-49d2-bb2f-49a3b4d241bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:14.015481 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9bf1f548-66fc-49d5-9976-1378482dbdfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6a00e39073343df9a8a0ceffb9c13d2', 'request_id': 'req-9bf1f548-66fc-49d5-9976-1378482dbdfb'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:14.021756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9bf1f548-66fc-49d5-9976-1378482dbdfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6a00e39073343df9a8a0ceffb9c13d2', 'request_id': 'req-9bf1f548-66fc-49d5-9976-1378482dbdfb'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:14.029813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9bf1f548-66fc-49d5-9976-1378482dbdfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6a00e39073343df9a8a0ceffb9c13d2', 'request_id': 'req-9bf1f548-66fc-49d5-9976-1378482dbdfb'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:14.041750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9bf1f548-66fc-49d5-9976-1378482dbdfb admin admin] The token cache is being invalidate because project b6a00e39073343df9a8a0ceffb9c13d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:39:14.042040 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9bf1f548-66fc-49d5-9976-1378482dbdfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:14.042693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9bf1f548-66fc-49d5-9976-1378482dbdfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:14.044667 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1006/2011] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:39:13 2022] DELETE /identity/v3/projects/b6a00e39073343df9a8a0ceffb9c13d2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:14.045449 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1f548-66fc-49d5-9976-1378482dbdfb admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:14.046777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-9bf1f548-66fc-49d5-9976-1378482dbdfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:14.052435 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:14.068934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1006/2012] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:14.069757 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff85652-c234-49d2-bb2f-49a3b4d241bd None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:14.071015 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fff85652-c234-49d2-bb2f-49a3b4d241bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:14.076382 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:14.084482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff85652-c234-49d2-bb2f-49a3b4d241bd None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:14.089362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83d64e-530d-48d2-9612-252a61a9b5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:14.090292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83d64e-530d-48d2-9612-252a61a9b5fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:14.090571 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83d64e-530d-48d2-9612-252a61a9b5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:14.090854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83d64e-530d-48d2-9612-252a61a9b5fe admin admin] PATH_INFO: `/v3/users/4dcf997b051b4003b70a477d419ad9d6` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:14.092097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83d64e-530d-48d2-9612-252a61a9b5fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dcf997b051b4003b70a477d419ad9d6)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:14.093975 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83d64e-530d-48d2-9612-252a61a9b5fe admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:14.128020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1f548-66fc-49d5-9976-1378482dbdfb admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:14.133009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f001b-3122-4241-bf89-98bcb51e42c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:14.133956 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f001b-3122-4241-bf89-98bcb51e42c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:14.134245 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f001b-3122-4241-bf89-98bcb51e42c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:14.134660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f001b-3122-4241-bf89-98bcb51e42c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:14.135354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f001b-3122-4241-bf89-98bcb51e42c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:14.137907 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f001b-3122-4241-bf89-98bcb51e42c6 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:14.144007 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-cf83d64e-530d-48d2-9612-252a61a9b5fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dcf997b051b4003b70a477d419ad9d6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:14.151338 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-cf83d64e-530d-48d2-9612-252a61a9b5fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dcf997b051b4003b70a477d419ad9d6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:14.152052 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-cf83d64e-530d-48d2-9612-252a61a9b5fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dcf997b051b4003b70a477d419ad9d6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:14.152930 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-cf83d64e-530d-48d2-9612-252a61a9b5fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dcf997b051b4003b70a477d419ad9d6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:14.165930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1007/2013] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:39:14 2022] POST /identity/v3/projects => generated 365 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:14.166171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-cf83d64e-530d-48d2-9612-252a61a9b5fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dcf997b051b4003b70a477d419ad9d6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:14.166744 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f001b-3122-4241-bf89-98bcb51e42c6 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:14.169002 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-b80f001b-3122-4241-bf89-98bcb51e42c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:14.180594 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1007/2014] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:39:14 2022] DELETE /identity/v3/users/4dcf997b051b4003b70a477d419ad9d6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:14.181394 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83d64e-530d-48d2-9612-252a61a9b5fe admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:14.182698 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-cf83d64e-530d-48d2-9612-252a61a9b5fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:14.234509 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f001b-3122-4241-bf89-98bcb51e42c6 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:14.254957 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83d64e-530d-48d2-9612-252a61a9b5fe admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:14.260093 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ce218-2d0b-45dd-a33c-12eb240047b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:14.260875 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ce218-2d0b-45dd-a33c-12eb240047b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:14.261265 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ce218-2d0b-45dd-a33c-12eb240047b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:14.262250 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ce218-2d0b-45dd-a33c-12eb240047b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:14.263275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ce218-2d0b-45dd-a33c-12eb240047b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:14.265285 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ce218-2d0b-45dd-a33c-12eb240047b0 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:14.300066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1008/2015] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:39:14 2022] POST /identity/v3/users => generated 283 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:14.303629 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582fd125-60b1-4d4d-8584-13b594456315 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:14.303629 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582fd125-60b1-4d4d-8584-13b594456315 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:14.303924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ce218-2d0b-45dd-a33c-12eb240047b0 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:14.304001 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582fd125-60b1-4d4d-8584-13b594456315 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:14.304001 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582fd125-60b1-4d4d-8584-13b594456315 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:14.305549 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-582fd125-60b1-4d4d-8584-13b594456315 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:14.314520 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ce218-2d0b-45dd-a33c-12eb240047b0 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:14.319463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c83be-d2ea-443c-b9e3-54eff1c006e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:14.320155 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c83be-d2ea-443c-b9e3-54eff1c006e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:14.320437 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c83be-d2ea-443c-b9e3-54eff1c006e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:14.320710 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c83be-d2ea-443c-b9e3-54eff1c006e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:14.321431 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c83be-d2ea-443c-b9e3-54eff1c006e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:14.323149 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582fd125-60b1-4d4d-8584-13b594456315 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:14.325433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c83be-d2ea-443c-b9e3-54eff1c006e7 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:14.325906 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582fd125-60b1-4d4d-8584-13b594456315 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:14.337289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1008/2016] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:39:14.339238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1009/2017] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:14.340904 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582fd125-60b1-4d4d-8584-13b594456315 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:14.366099 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582fd125-60b1-4d4d-8584-13b594456315 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:14.372800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0eabcb-861c-4eee-b1e6-4a94c7c89c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:14.373848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0eabcb-861c-4eee-b1e6-4a94c7c89c0d admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:14.377006 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0eabcb-861c-4eee-b1e6-4a94c7c89c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:14.377006 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0eabcb-861c-4eee-b1e6-4a94c7c89c0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:14.377006 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0eabcb-861c-4eee-b1e6-4a94c7c89c0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:14.377539 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0eabcb-861c-4eee-b1e6-4a94c7c89c0d admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:14.391220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1009/2018] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:14.398933 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c83be-d2ea-443c-b9e3-54eff1c006e7 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:14.410233 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c83be-d2ea-443c-b9e3-54eff1c006e7 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:14.416212 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066617d4-4065-4047-94bd-ce4af9c7fa81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:14.417315 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066617d4-4065-4047-94bd-ce4af9c7fa81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:14.417600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066617d4-4065-4047-94bd-ce4af9c7fa81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:14.417910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066617d4-4065-4047-94bd-ce4af9c7fa81 admin admin] PATH_INFO: `/v3/projects/69001a078435420ca20c70e406c217bf/users/112bd2b1bae64be699cfe0e5a5aede52/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:14.437431 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066617d4-4065-4047-94bd-ce4af9c7fa81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69001a078435420ca20c70e406c217bf, user_id=112bd2b1bae64be699cfe0e5a5aede52, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:14.440654 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066617d4-4065-4047-94bd-ce4af9c7fa81 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:14.467615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1010/2019] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:14 2022] PUT /identity/v3/projects/69001a078435420ca20c70e406c217bf/users/112bd2b1bae64be699cfe0e5a5aede52/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:14.469814 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20045f82-671a-438a-a8f3-95a61b0c633f None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:14.470347 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20045f82-671a-438a-a8f3-95a61b0c633f None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:14.470823 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20045f82-671a-438a-a8f3-95a61b0c633f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:14.516335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-20045f82-671a-438a-a8f3-95a61b0c633f None None] MFA Rules not processed for user `112bd2b1bae64be699cfe0e5a5aede52`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:14.518055 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-20045f82-671a-438a-a8f3-95a61b0c633f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:14.582922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:14.616400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1010/2020] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:39:14.628669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066617d4-4065-4047-94bd-ce4af9c7fa81 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:14.629975 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20045f82-671a-438a-a8f3-95a61b0c633f None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:14.630877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-066617d4-4065-4047-94bd-ce4af9c7fa81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:14.631394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-20045f82-671a-438a-a8f3-95a61b0c633f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:14.671372 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:14.728125 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066617d4-4065-4047-94bd-ce4af9c7fa81 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:14.732999 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5d78d-7d09-414c-9212-cec509580851 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:14.733723 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5d78d-7d09-414c-9212-cec509580851 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:14.734051 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5d78d-7d09-414c-9212-cec509580851 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:14.734333 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5d78d-7d09-414c-9212-cec509580851 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:14.744269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5d78d-7d09-414c-9212-cec509580851 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:14.745594 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5d78d-7d09-414c-9212-cec509580851 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:14.756087 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20045f82-671a-438a-a8f3-95a61b0c633f None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:14.757442 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1011/2021] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:14 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:39:14.761227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed64c2-b3e4-49b7-93ce-b4124956075e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:14.763002 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed64c2-b3e4-49b7-93ce-b4124956075e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:14.763470 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed64c2-b3e4-49b7-93ce-b4124956075e admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:14.763711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed64c2-b3e4-49b7-93ce-b4124956075e admin admin] PATH_INFO: `/v3/projects/111c20f9e83941e8aa4baecd6b3369e0` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:14.765089 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed64c2-b3e4-49b7-93ce-b4124956075e admin admin] RBAC: Authorizing `identity:delete_project(project_id=111c20f9e83941e8aa4baecd6b3369e0)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:14.767139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed64c2-b3e4-49b7-93ce-b4124956075e admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:14.842483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-79ed64c2-b3e4-49b7-93ce-b4124956075e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '111c20f9e83941e8aa4baecd6b3369e0', 'request_id': 'req-79ed64c2-b3e4-49b7-93ce-b4124956075e'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:14.851720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-79ed64c2-b3e4-49b7-93ce-b4124956075e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '111c20f9e83941e8aa4baecd6b3369e0', 'request_id': 'req-79ed64c2-b3e4-49b7-93ce-b4124956075e'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:14.867702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-79ed64c2-b3e4-49b7-93ce-b4124956075e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '111c20f9e83941e8aa4baecd6b3369e0', 'request_id': 'req-79ed64c2-b3e4-49b7-93ce-b4124956075e'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:14.888671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-79ed64c2-b3e4-49b7-93ce-b4124956075e admin admin] The token cache is being invalidate because project 111c20f9e83941e8aa4baecd6b3369e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:39:14.889342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-79ed64c2-b3e4-49b7-93ce-b4124956075e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:14.890656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-79ed64c2-b3e4-49b7-93ce-b4124956075e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:14.893010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1011/2022] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:39:14 2022] DELETE /identity/v3/projects/111c20f9e83941e8aa4baecd6b3369e0 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:14.900543 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f1b36-dcee-43d9-b5b9-8375fe9d0738 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:14.900867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f1b36-dcee-43d9-b5b9-8375fe9d0738 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:14.901487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f1b36-dcee-43d9-b5b9-8375fe9d0738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:14.927586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b15f1b36-dcee-43d9-b5b9-8375fe9d0738 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:14.929692 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b15f1b36-dcee-43d9-b5b9-8375fe9d0738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:15.019607 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1012/2023] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:15.023373 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f6da5-3279-49c3-8cde-c5c2f630abbf None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:15.023753 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f6da5-3279-49c3-8cde-c5c2f630abbf None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:15.024575 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f6da5-3279-49c3-8cde-c5c2f630abbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:15.048181 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-a64f6da5-3279-49c3-8cde-c5c2f630abbf None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:15.049412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a64f6da5-3279-49c3-8cde-c5c2f630abbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:15.064054 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1012/2024] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:39:15.066034 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f1b36-dcee-43d9-b5b9-8375fe9d0738 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:15.091326 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f1b36-dcee-43d9-b5b9-8375fe9d0738 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:15.096608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36f6b1-3f9c-42f4-bde2-3ae2a39c723e None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:15.096694 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36f6b1-3f9c-42f4-bde2-3ae2a39c723e None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:15.097215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36f6b1-3f9c-42f4-bde2-3ae2a39c723e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:15.104239 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2f8eb-27de-4e26-afe4-503e1caff014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:15.104924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2f8eb-27de-4e26-afe4-503e1caff014 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:15.107304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2f8eb-27de-4e26-afe4-503e1caff014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:15.107600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2f8eb-27de-4e26-afe4-503e1caff014 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:15.108389 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d2f8eb-27de-4e26-afe4-503e1caff014 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:39:15.108701 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d2f8eb-27de-4e26-afe4-503e1caff014 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:15.110519 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d2f8eb-27de-4e26-afe4-503e1caff014 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:15.128404 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1013/2025] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:39:15.129765 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2f8eb-27de-4e26-afe4-503e1caff014 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:15.137381 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-4a36f6b1-3f9c-42f4-bde2-3ae2a39c723e None None] MFA Rules not processed for user `446b13b4661542c097afafb32443dae4`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:15.139244 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-4a36f6b1-3f9c-42f4-bde2-3ae2a39c723e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:15.139974 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2f8eb-27de-4e26-afe4-503e1caff014 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:15.145275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87ff66-6a8c-4bf5-bed0-4f41e066531a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:15.145944 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87ff66-6a8c-4bf5-bed0-4f41e066531a admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:15.146230 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87ff66-6a8c-4bf5-bed0-4f41e066531a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:15.146579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87ff66-6a8c-4bf5-bed0-4f41e066531a admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:15.147280 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f87ff66-6a8c-4bf5-bed0-4f41e066531a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:15.149546 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f87ff66-6a8c-4bf5-bed0-4f41e066531a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:15.183289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1014/2026] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:39:15 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:15.188275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87ff66-6a8c-4bf5-bed0-4f41e066531a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:15.242151 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:15.267953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1013/2027] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:39:15.271271 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36f6b1-3f9c-42f4-bde2-3ae2a39c723e None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:15.273658 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-4a36f6b1-3f9c-42f4-bde2-3ae2a39c723e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:15.279679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87ff66-6a8c-4bf5-bed0-4f41e066531a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:15.285598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007a454-4027-48c7-bcf0-74ae7e3ad085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:15.286280 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007a454-4027-48c7-bcf0-74ae7e3ad085 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:15.287168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007a454-4027-48c7-bcf0-74ae7e3ad085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:15.287168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007a454-4027-48c7-bcf0-74ae7e3ad085 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:15.287522 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6007a454-4027-48c7-bcf0-74ae7e3ad085 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:15.289939 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6007a454-4027-48c7-bcf0-74ae7e3ad085 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:15.318154 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1015/2028] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:39:15 2022] POST /identity/v3/users => generated 283 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:39:15.319287 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007a454-4027-48c7-bcf0-74ae7e3ad085 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:15.329777 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007a454-4027-48c7-bcf0-74ae7e3ad085 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:15.334695 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba0757-89e7-48c1-84ca-4ff9de64c7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:15.335360 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba0757-89e7-48c1-84ca-4ff9de64c7d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:15.335642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba0757-89e7-48c1-84ca-4ff9de64c7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:15.335990 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba0757-89e7-48c1-84ca-4ff9de64c7d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:15.336743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba0757-89e7-48c1-84ca-4ff9de64c7d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:15.338380 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba0757-89e7-48c1-84ca-4ff9de64c7d4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:15.347320 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36f6b1-3f9c-42f4-bde2-3ae2a39c723e None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:15.349383 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1016/2029] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:39:15.353129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba0757-89e7-48c1-84ca-4ff9de64c7d4 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:15.364201 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba0757-89e7-48c1-84ca-4ff9de64c7d4 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:15.369031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d350744-b94e-4571-89aa-24b5ac020316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:15.369803 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d350744-b94e-4571-89aa-24b5ac020316 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:15.370088 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d350744-b94e-4571-89aa-24b5ac020316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:15.370361 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d350744-b94e-4571-89aa-24b5ac020316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:15.371232 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d350744-b94e-4571-89aa-24b5ac020316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:15.372847 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d350744-b94e-4571-89aa-24b5ac020316 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:15.374938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:15.388794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1017/2030] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:39:15.390651 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d350744-b94e-4571-89aa-24b5ac020316 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:15.400665 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d350744-b94e-4571-89aa-24b5ac020316 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:15.407166 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db44848-3a06-4ffe-bee6-fc6125588a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:15.408230 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db44848-3a06-4ffe-bee6-fc6125588a47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:15.408516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db44848-3a06-4ffe-bee6-fc6125588a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:15.408793 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db44848-3a06-4ffe-bee6-fc6125588a47 admin admin] PATH_INFO: `/v3/projects/c2e8f3ca5d224847a240caa5a090b025/users/812c09be835944fc85c041ae93f990cd/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:15.413920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e91f6-ef0c-4767-b9b4-5c87cd09f815 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:15.414562 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e91f6-ef0c-4767-b9b4-5c87cd09f815 service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:15.414844 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e91f6-ef0c-4767-b9b4-5c87cd09f815 service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:15.415118 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e91f6-ef0c-4767-b9b4-5c87cd09f815 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:15.424809 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db44848-3a06-4ffe-bee6-fc6125588a47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e8f3ca5d224847a240caa5a090b025, user_id=812c09be835944fc85c041ae93f990cd, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:15.427485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db44848-3a06-4ffe-bee6-fc6125588a47 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:15.440554 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1018/2031] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:15 2022] PUT /identity/v3/projects/c2e8f3ca5d224847a240caa5a090b025/users/812c09be835944fc85c041ae93f990cd/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:15.450649 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d139ac6-4aed-441e-84d4-d5fe4d0b13fa None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:15.450934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d139ac6-4aed-441e-84d4-d5fe4d0b13fa None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:15.451238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d139ac6-4aed-441e-84d4-d5fe4d0b13fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:15.476484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e91f6-ef0c-4767-b9b4-5c87cd09f815 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:15.477911 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e91f6-ef0c-4767-b9b4-5c87cd09f815 service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:15.478383 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-0d139ac6-4aed-441e-84d4-d5fe4d0b13fa None None] MFA Rules not processed for user `812c09be835944fc85c041ae93f990cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:15.479619 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0d139ac6-4aed-441e-84d4-d5fe4d0b13fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:15.487529 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1014/2032] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:39:15.546676 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1019/2033] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:15 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:39:15.596889 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e91f6-ef0c-4767-b9b4-5c87cd09f815 service cinder] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:15.599195 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-1d2e91f6-ef0c-4767-b9b4-5c87cd09f815 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:15.610631 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e91f6-ef0c-4767-b9b4-5c87cd09f815 service cinder] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:15.688329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270e459-32bf-48ef-b70f-be23fef96471 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:15.688990 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270e459-32bf-48ef-b70f-be23fef96471 service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:15.689479 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270e459-32bf-48ef-b70f-be23fef96471 service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:15.689766 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270e459-32bf-48ef-b70f-be23fef96471 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:15.701152 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270e459-32bf-48ef-b70f-be23fef96471 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:15.702685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270e459-32bf-48ef-b70f-be23fef96471 service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:15.713011 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1015/2034] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:39:18.952343 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d015a5-28a9-4c2b-911f-09e361ae36b9 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:18.952878 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d015a5-28a9-4c2b-911f-09e361ae36b9 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:18.952956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d015a5-28a9-4c2b-911f-09e361ae36b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:18.973059 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b6d015a5-28a9-4c2b-911f-09e361ae36b9 None None] MFA Rules not processed for user `446b13b4661542c097afafb32443dae4`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:18.974248 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b6d015a5-28a9-4c2b-911f-09e361ae36b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:19.043972 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1020/2035] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:24.362337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270e459-32bf-48ef-b70f-be23fef96471 service cinder] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:24.364369 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a270e459-32bf-48ef-b70f-be23fef96471 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:24.376924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:24.449330 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270e459-32bf-48ef-b70f-be23fef96471 service cinder] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:24.454243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa90c0-de4b-4f39-b8e9-6624d0c1ac24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:24.454985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa90c0-de4b-4f39-b8e9-6624d0c1ac24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:24.455276 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa90c0-de4b-4f39-b8e9-6624d0c1ac24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:24.456022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa90c0-de4b-4f39-b8e9-6624d0c1ac24 admin admin] PATH_INFO: `/v3/users/79db19df04eb4bd38f967015efcd843c` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:24.456776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa90c0-de4b-4f39-b8e9-6624d0c1ac24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79db19df04eb4bd38f967015efcd843c)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:24.459611 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa90c0-de4b-4f39-b8e9-6624d0c1ac24 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:24.522482 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-bdaa90c0-de4b-4f39-b8e9-6624d0c1ac24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79db19df04eb4bd38f967015efcd843c'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:24.530469 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-bdaa90c0-de4b-4f39-b8e9-6624d0c1ac24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79db19df04eb4bd38f967015efcd843c'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:24.538760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-bdaa90c0-de4b-4f39-b8e9-6624d0c1ac24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79db19df04eb4bd38f967015efcd843c'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:24.539400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-bdaa90c0-de4b-4f39-b8e9-6624d0c1ac24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79db19df04eb4bd38f967015efcd843c'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:24.551769 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-bdaa90c0-de4b-4f39-b8e9-6624d0c1ac24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79db19df04eb4bd38f967015efcd843c'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:24.559192 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1016/2036] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:39:24 2022] DELETE /identity/v3/users/79db19df04eb4bd38f967015efcd843c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:24.563464 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2bc63-bc72-4db8-8ff2-5ae836741299 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:24.563945 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2bc63-bc72-4db8-8ff2-5ae836741299 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:24.564084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2bc63-bc72-4db8-8ff2-5ae836741299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:24.582823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-13f2bc63-bc72-4db8-8ff2-5ae836741299 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:24.584050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-13f2bc63-bc72-4db8-8ff2-5ae836741299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:24.662703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1021/2037] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:24.674412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa90c0-de4b-4f39-b8e9-6624d0c1ac24 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:24.676129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-bdaa90c0-de4b-4f39-b8e9-6624d0c1ac24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:24.746591 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa90c0-de4b-4f39-b8e9-6624d0c1ac24 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:24.753843 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563dff48-4070-49ec-ae0f-31e125703efa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:24.754542 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563dff48-4070-49ec-ae0f-31e125703efa service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:24.754852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563dff48-4070-49ec-ae0f-31e125703efa service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:24.755137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563dff48-4070-49ec-ae0f-31e125703efa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:24.767099 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563dff48-4070-49ec-ae0f-31e125703efa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:24.768723 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563dff48-4070-49ec-ae0f-31e125703efa service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:24.782561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1017/2038] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:39:25.257243 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2bc63-bc72-4db8-8ff2-5ae836741299 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:25.265072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-13f2bc63-bc72-4db8-8ff2-5ae836741299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:25.301193 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2bc63-bc72-4db8-8ff2-5ae836741299 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:25.315290 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53814d4-d458-4285-81b6-b786e9d2580a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:25.319015 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53814d4-d458-4285-81b6-b786e9d2580a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:25.319605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53814d4-d458-4285-81b6-b786e9d2580a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:25.320849 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53814d4-d458-4285-81b6-b786e9d2580a admin admin] PATH_INFO: `/v3/projects/7a5149b487a845ec8a15122a2531fad3` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:25.325287 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53814d4-d458-4285-81b6-b786e9d2580a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a5149b487a845ec8a15122a2531fad3)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:25.331919 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53814d4-d458-4285-81b6-b786e9d2580a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:25.348063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:25.473596 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a53814d4-d458-4285-81b6-b786e9d2580a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a5149b487a845ec8a15122a2531fad3', 'request_id': 'req-a53814d4-d458-4285-81b6-b786e9d2580a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:25.483998 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a53814d4-d458-4285-81b6-b786e9d2580a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a5149b487a845ec8a15122a2531fad3', 'request_id': 'req-a53814d4-d458-4285-81b6-b786e9d2580a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:25.493973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a53814d4-d458-4285-81b6-b786e9d2580a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a5149b487a845ec8a15122a2531fad3', 'request_id': 'req-a53814d4-d458-4285-81b6-b786e9d2580a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:25.508353 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a53814d4-d458-4285-81b6-b786e9d2580a admin admin] The token cache is being invalidate because project 7a5149b487a845ec8a15122a2531fad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:39:25.509746 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a53814d4-d458-4285-81b6-b786e9d2580a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:25.510718 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a53814d4-d458-4285-81b6-b786e9d2580a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:25.513135 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1022/2039] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:39:25 2022] DELETE /identity/v3/projects/7a5149b487a845ec8a15122a2531fad3 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:25.517197 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563dff48-4070-49ec-ae0f-31e125703efa service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:25.530080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563dff48-4070-49ec-ae0f-31e125703efa service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:25.568136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:25.615400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7adb358-9c7d-4b2d-baa9-799a5cce9bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:25.616612 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7adb358-9c7d-4b2d-baa9-799a5cce9bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:25.617096 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7adb358-9c7d-4b2d-baa9-799a5cce9bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:25.617329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7adb358-9c7d-4b2d-baa9-799a5cce9bae admin admin] PATH_INFO: `/v3/users/726f8a21a9a94192a737c6ce2eb1a5cf` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:25.622905 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7adb358-9c7d-4b2d-baa9-799a5cce9bae admin admin] RBAC: Authorizing `identity:delete_user(user_id=726f8a21a9a94192a737c6ce2eb1a5cf)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:25.627828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7adb358-9c7d-4b2d-baa9-799a5cce9bae admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:25.718994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e7adb358-9c7d-4b2d-baa9-799a5cce9bae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '726f8a21a9a94192a737c6ce2eb1a5cf'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:25.733206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e7adb358-9c7d-4b2d-baa9-799a5cce9bae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '726f8a21a9a94192a737c6ce2eb1a5cf'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:25.745179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e7adb358-9c7d-4b2d-baa9-799a5cce9bae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '726f8a21a9a94192a737c6ce2eb1a5cf'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:25.745981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e7adb358-9c7d-4b2d-baa9-799a5cce9bae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '726f8a21a9a94192a737c6ce2eb1a5cf'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:25.769676 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e7adb358-9c7d-4b2d-baa9-799a5cce9bae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '726f8a21a9a94192a737c6ce2eb1a5cf'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:25.779587 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1018/2040] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:39:25 2022] DELETE /identity/v3/users/726f8a21a9a94192a737c6ce2eb1a5cf => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:39:26.149196 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53814d4-d458-4285-81b6-b786e9d2580a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:26.150874 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a53814d4-d458-4285-81b6-b786e9d2580a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:26.173539 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53814d4-d458-4285-81b6-b786e9d2580a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:26.237507 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45004d1-b9e0-489f-83d2-90abe723d800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:26.238381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45004d1-b9e0-489f-83d2-90abe723d800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:26.238663 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45004d1-b9e0-489f-83d2-90abe723d800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:26.238965 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45004d1-b9e0-489f-83d2-90abe723d800 admin admin] PATH_INFO: `/v3/projects/3025ef996c1b48f2b8f925dae544bfeb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:26.240173 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45004d1-b9e0-489f-83d2-90abe723d800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3025ef996c1b48f2b8f925dae544bfeb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:26.242538 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45004d1-b9e0-489f-83d2-90abe723d800 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:26.292167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c45004d1-b9e0-489f-83d2-90abe723d800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3025ef996c1b48f2b8f925dae544bfeb', 'request_id': 'req-c45004d1-b9e0-489f-83d2-90abe723d800'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:26.300019 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c45004d1-b9e0-489f-83d2-90abe723d800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3025ef996c1b48f2b8f925dae544bfeb', 'request_id': 'req-c45004d1-b9e0-489f-83d2-90abe723d800'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:26.305619 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c45004d1-b9e0-489f-83d2-90abe723d800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3025ef996c1b48f2b8f925dae544bfeb', 'request_id': 'req-c45004d1-b9e0-489f-83d2-90abe723d800'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:26.318657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c45004d1-b9e0-489f-83d2-90abe723d800 admin admin] The token cache is being invalidate because project 3025ef996c1b48f2b8f925dae544bfeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:39:26.318942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c45004d1-b9e0-489f-83d2-90abe723d800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:26.319600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c45004d1-b9e0-489f-83d2-90abe723d800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:26.321182 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1023/2041] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:39:26 2022] DELETE /identity/v3/projects/3025ef996c1b48f2b8f925dae544bfeb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:26.326057 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d05d0-387b-430d-ac5b-af126c54bada None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:26.326347 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d05d0-387b-430d-ac5b-af126c54bada None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:26.326623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d05d0-387b-430d-ac5b-af126c54bada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:26.344359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-550d05d0-387b-430d-ac5b-af126c54bada None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:26.345560 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-550d05d0-387b-430d-ac5b-af126c54bada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:26.419288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1019/2042] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:26.427007 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcbcf9-b085-408f-8a81-26b08ec79e37 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:26.427368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcbcf9-b085-408f-8a81-26b08ec79e37 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:26.427650 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcbcf9-b085-408f-8a81-26b08ec79e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:26.447571 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-d9fcbcf9-b085-408f-8a81-26b08ec79e37 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:26.449055 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d9fcbcf9-b085-408f-8a81-26b08ec79e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:26.459284 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1024/2043] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:39:26.463635 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d05d0-387b-430d-ac5b-af126c54bada None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:26.473617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d05d0-387b-430d-ac5b-af126c54bada None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:26.479139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d33f1-55a4-4bc3-816b-9af421918dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:26.480702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d33f1-55a4-4bc3-816b-9af421918dcb admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:26.481020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d33f1-55a4-4bc3-816b-9af421918dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:26.481363 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d33f1-55a4-4bc3-816b-9af421918dcb admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:26.482679 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d33f1-55a4-4bc3-816b-9af421918dcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:39:26.483198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d33f1-55a4-4bc3-816b-9af421918dcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:26.486302 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d33f1-55a4-4bc3-816b-9af421918dcb admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:26.508962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1020/2044] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:39:26.510772 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcbcf9-b085-408f-8a81-26b08ec79e37 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:26.524145 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcbcf9-b085-408f-8a81-26b08ec79e37 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:26.531368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496ec87-eff5-4608-84a6-b69a869aafd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:26.532156 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496ec87-eff5-4608-84a6-b69a869aafd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:26.532507 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496ec87-eff5-4608-84a6-b69a869aafd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:26.532852 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496ec87-eff5-4608-84a6-b69a869aafd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:26.533642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496ec87-eff5-4608-84a6-b69a869aafd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:26.535624 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496ec87-eff5-4608-84a6-b69a869aafd9 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:26.553392 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:26.577322 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1025/2045] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:39:26 2022] POST /identity/v3/projects => generated 365 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:26.581152 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d33f1-55a4-4bc3-816b-9af421918dcb admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:26.597772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d33f1-55a4-4bc3-816b-9af421918dcb admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:26.649322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:26.681675 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a3082-c522-465c-8198-6bfa92eeafa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:26.682677 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a3082-c522-465c-8198-6bfa92eeafa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:26.682819 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a3082-c522-465c-8198-6bfa92eeafa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:26.684562 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a3082-c522-465c-8198-6bfa92eeafa3 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:26.685499 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a3082-c522-465c-8198-6bfa92eeafa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:26.687543 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a3082-c522-465c-8198-6bfa92eeafa3 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:26.719440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1021/2046] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:39:26 2022] POST /identity/v3/users => generated 283 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:26.724086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496ec87-eff5-4608-84a6-b69a869aafd9 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:26.735417 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496ec87-eff5-4608-84a6-b69a869aafd9 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:26.744357 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8ac01-4be3-4653-8201-243d1c8b82ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:26.745601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8ac01-4be3-4653-8201-243d1c8b82ed admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:26.746386 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8ac01-4be3-4653-8201-243d1c8b82ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:26.746922 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8ac01-4be3-4653-8201-243d1c8b82ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:26.747660 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8ac01-4be3-4653-8201-243d1c8b82ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:26.749854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8ac01-4be3-4653-8201-243d1c8b82ed admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:26.764170 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1026/2047] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:26.765851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a3082-c522-465c-8198-6bfa92eeafa3 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:26.780627 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a3082-c522-465c-8198-6bfa92eeafa3 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:26.786996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a101e1-d35a-403e-842b-e119265f2d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:26.787777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a101e1-d35a-403e-842b-e119265f2d84 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:26.788123 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a101e1-d35a-403e-842b-e119265f2d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:26.788432 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a101e1-d35a-403e-842b-e119265f2d84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:26.789157 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a101e1-d35a-403e-842b-e119265f2d84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:26.790760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a101e1-d35a-403e-842b-e119265f2d84 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:26.803554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1022/2048] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:26.808124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8ac01-4be3-4653-8201-243d1c8b82ed admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:26.818413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8ac01-4be3-4653-8201-243d1c8b82ed admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:26.823215 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cd02e-6376-46e8-931f-15b9d3f80024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:26.824738 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cd02e-6376-46e8-931f-15b9d3f80024 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:26.825136 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cd02e-6376-46e8-931f-15b9d3f80024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:26.825525 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cd02e-6376-46e8-931f-15b9d3f80024 admin admin] PATH_INFO: `/v3/projects/056fbc9190d54341bc2a92f3c608f409/users/c6ca3919ead04465bc8e859e5b81d325/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:26.843001 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cd02e-6376-46e8-931f-15b9d3f80024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=056fbc9190d54341bc2a92f3c608f409, user_id=c6ca3919ead04465bc8e859e5b81d325, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:26.846262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cd02e-6376-46e8-931f-15b9d3f80024 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:26.874157 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1027/2049] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:26 2022] PUT /identity/v3/projects/056fbc9190d54341bc2a92f3c608f409/users/c6ca3919ead04465bc8e859e5b81d325/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:26.885889 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f7367-7efe-4424-a5b6-85d86ce60b9c None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:26.886766 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f7367-7efe-4424-a5b6-85d86ce60b9c None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:26.886958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f7367-7efe-4424-a5b6-85d86ce60b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:26.919453 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-068f7367-7efe-4424-a5b6-85d86ce60b9c None None] MFA Rules not processed for user `c6ca3919ead04465bc8e859e5b81d325`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:26.920715 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-068f7367-7efe-4424-a5b6-85d86ce60b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:27.015330 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1023/2050] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:26 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:27.020135 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cd02e-6376-46e8-931f-15b9d3f80024 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:27.101698 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cd02e-6376-46e8-931f-15b9d3f80024 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:27.108907 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c38d2a-e76a-4bc7-81de-7d1ed3a3569b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:27.109567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c38d2a-e76a-4bc7-81de-7d1ed3a3569b admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:27.109996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c38d2a-e76a-4bc7-81de-7d1ed3a3569b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:27.110270 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c38d2a-e76a-4bc7-81de-7d1ed3a3569b admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:27.110979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c38d2a-e76a-4bc7-81de-7d1ed3a3569b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:27.112906 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c38d2a-e76a-4bc7-81de-7d1ed3a3569b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:27.146550 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1028/2051] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:39:27 2022] POST /identity/v3/projects => generated 363 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:39:27.148465 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f7367-7efe-4424-a5b6-85d86ce60b9c None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:27.164532 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f7367-7efe-4424-a5b6-85d86ce60b9c None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:27.242430 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1cbf3-3526-4f28-8c09-348090c2eb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:27.242783 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1cbf3-3526-4f28-8c09-348090c2eb16 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:27.244036 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1cbf3-3526-4f28-8c09-348090c2eb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:27.244442 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1cbf3-3526-4f28-8c09-348090c2eb16 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:27.245343 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1cbf3-3526-4f28-8c09-348090c2eb16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:27.248419 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1cbf3-3526-4f28-8c09-348090c2eb16 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:27.280473 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1024/2052] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:39:27 2022] POST /identity/v3/users => generated 288 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:27.281291 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c38d2a-e76a-4bc7-81de-7d1ed3a3569b admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:27.294289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c38d2a-e76a-4bc7-81de-7d1ed3a3569b admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:27.299279 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7390164-8389-4ea6-9bee-7c41aafb8e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:27.300293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7390164-8389-4ea6-9bee-7c41aafb8e8c admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:27.300804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7390164-8389-4ea6-9bee-7c41aafb8e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:27.301282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7390164-8389-4ea6-9bee-7c41aafb8e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:27.302216 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7390164-8389-4ea6-9bee-7c41aafb8e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:27.304122 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7390164-8389-4ea6-9bee-7c41aafb8e8c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:27.316422 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1029/2053] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:27.321131 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1cbf3-3526-4f28-8c09-348090c2eb16 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:27.333298 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1cbf3-3526-4f28-8c09-348090c2eb16 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:27.340944 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde318d3-b38b-4541-bd1a-e13c7b42f033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:27.341841 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde318d3-b38b-4541-bd1a-e13c7b42f033 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:27.342309 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde318d3-b38b-4541-bd1a-e13c7b42f033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:27.342628 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde318d3-b38b-4541-bd1a-e13c7b42f033 admin admin] PATH_INFO: `/v3/projects/ba81c02d82514a3bb2a4388d1a1d3344/users/575d93a5ba78440c86e98d2a863ac372/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:27.361375 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde318d3-b38b-4541-bd1a-e13c7b42f033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba81c02d82514a3bb2a4388d1a1d3344, user_id=575d93a5ba78440c86e98d2a863ac372, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:27.364062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde318d3-b38b-4541-bd1a-e13c7b42f033 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:27.381162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1025/2054] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:27 2022] PUT /identity/v3/projects/ba81c02d82514a3bb2a4388d1a1d3344/users/575d93a5ba78440c86e98d2a863ac372/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:27.384881 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7390164-8389-4ea6-9bee-7c41aafb8e8c admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:27.399214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7390164-8389-4ea6-9bee-7c41aafb8e8c admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:27.457090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5d7a5-4fc6-402b-be2e-c3fd436b4d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:27.457769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5d7a5-4fc6-402b-be2e-c3fd436b4d14 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:27.458038 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5d7a5-4fc6-402b-be2e-c3fd436b4d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:27.458311 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5d7a5-4fc6-402b-be2e-c3fd436b4d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:27.459029 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5d7a5-4fc6-402b-be2e-c3fd436b4d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:27.460649 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5d7a5-4fc6-402b-be2e-c3fd436b4d14 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:27.470875 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1030/2055] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:27.475207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde318d3-b38b-4541-bd1a-e13c7b42f033 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:27.485857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde318d3-b38b-4541-bd1a-e13c7b42f033 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:27.491597 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf283dc-7513-4b7a-b250-8f24078049c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:27.492362 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf283dc-7513-4b7a-b250-8f24078049c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:27.492712 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf283dc-7513-4b7a-b250-8f24078049c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:27.492994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf283dc-7513-4b7a-b250-8f24078049c9 admin admin] PATH_INFO: `/v3/projects/ba81c02d82514a3bb2a4388d1a1d3344/users/575d93a5ba78440c86e98d2a863ac372/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:27.495066 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf283dc-7513-4b7a-b250-8f24078049c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba81c02d82514a3bb2a4388d1a1d3344, user_id=575d93a5ba78440c86e98d2a863ac372, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:27.498542 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf283dc-7513-4b7a-b250-8f24078049c9 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:27.501802 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:27.510514 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1026/2056] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:27 2022] PUT /identity/v3/projects/ba81c02d82514a3bb2a4388d1a1d3344/users/575d93a5ba78440c86e98d2a863ac372/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:39:27.523063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60756bf5-decc-4014-b4be-bdf0be30880b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:27.523362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60756bf5-decc-4014-b4be-bdf0be30880b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:27.523641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60756bf5-decc-4014-b4be-bdf0be30880b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:27.554192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-60756bf5-decc-4014-b4be-bdf0be30880b None None] MFA Rules not processed for user `575d93a5ba78440c86e98d2a863ac372`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:27.555409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-60756bf5-decc-4014-b4be-bdf0be30880b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:27.576363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:27.632318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1031/2057] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:27 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:27.645468 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf283dc-7513-4b7a-b250-8f24078049c9 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:27.647351 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-faf283dc-7513-4b7a-b250-8f24078049c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:27.714829 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf283dc-7513-4b7a-b250-8f24078049c9 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:27.718528 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2eb7b6-6236-4789-b77b-fa498d31a460 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:27.719340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2eb7b6-6236-4789-b77b-fa498d31a460 service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:27.719514 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2eb7b6-6236-4789-b77b-fa498d31a460 service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:27.719807 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2eb7b6-6236-4789-b77b-fa498d31a460 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:27.730122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2eb7b6-6236-4789-b77b-fa498d31a460 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:27.731744 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2eb7b6-6236-4789-b77b-fa498d31a460 service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:27.744298 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1027/2058] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:27 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:39:27.868847 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60756bf5-decc-4014-b4be-bdf0be30880b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:27.879349 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60756bf5-decc-4014-b4be-bdf0be30880b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:27.883830 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5959baa-d6e8-4cc9-9e97-6ca45cdc65a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:27.884549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5959baa-d6e8-4cc9-9e97-6ca45cdc65a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:27.884900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5959baa-d6e8-4cc9-9e97-6ca45cdc65a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:27.885240 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5959baa-d6e8-4cc9-9e97-6ca45cdc65a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:27.949689 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5959baa-d6e8-4cc9-9e97-6ca45cdc65a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:27.951216 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5959baa-d6e8-4cc9-9e97-6ca45cdc65a2 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:27.961978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1032/2059] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:27 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:39:31.742318 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2eb7b6-6236-4789-b77b-fa498d31a460 service cinder] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:31.805307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2eb7b6-6236-4789-b77b-fa498d31a460 service cinder] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:31.811033 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9dc1e-54de-4bfd-8f0f-1cc116cab2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:31.811943 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9dc1e-54de-4bfd-8f0f-1cc116cab2f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:31.812228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9dc1e-54de-4bfd-8f0f-1cc116cab2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:31.812504 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9dc1e-54de-4bfd-8f0f-1cc116cab2f0 admin admin] PATH_INFO: `/v3/users/c6ca3919ead04465bc8e859e5b81d325` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:31.813734 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9dc1e-54de-4bfd-8f0f-1cc116cab2f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6ca3919ead04465bc8e859e5b81d325)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:31.815678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9dc1e-54de-4bfd-8f0f-1cc116cab2f0 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:31.861566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-06b9dc1e-54de-4bfd-8f0f-1cc116cab2f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6ca3919ead04465bc8e859e5b81d325'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:31.868631 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-06b9dc1e-54de-4bfd-8f0f-1cc116cab2f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6ca3919ead04465bc8e859e5b81d325'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:31.876536 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-06b9dc1e-54de-4bfd-8f0f-1cc116cab2f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6ca3919ead04465bc8e859e5b81d325'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:31.877150 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-06b9dc1e-54de-4bfd-8f0f-1cc116cab2f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6ca3919ead04465bc8e859e5b81d325'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:31.889326 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-06b9dc1e-54de-4bfd-8f0f-1cc116cab2f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6ca3919ead04465bc8e859e5b81d325'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:31.895594 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1028/2060] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:39:31 2022] DELETE /identity/v3/users/c6ca3919ead04465bc8e859e5b81d325 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:31.900231 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa1366-032d-46b6-be3f-f8ca2483b179 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:31.900683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa1366-032d-46b6-be3f-f8ca2483b179 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:31.900792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa1366-032d-46b6-be3f-f8ca2483b179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:31.922099 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-aeaa1366-032d-46b6-be3f-f8ca2483b179 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:31.923312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-aeaa1366-032d-46b6-be3f-f8ca2483b179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:31.982783 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1033/2061] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:31.992491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9dc1e-54de-4bfd-8f0f-1cc116cab2f0 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:31.993925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-06b9dc1e-54de-4bfd-8f0f-1cc116cab2f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:32.017189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:32.063706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9dc1e-54de-4bfd-8f0f-1cc116cab2f0 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:32.067372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a537a7-c0d5-4ce3-971e-c7e583a8e175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:32.068192 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a537a7-c0d5-4ce3-971e-c7e583a8e175 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:32.068476 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a537a7-c0d5-4ce3-971e-c7e583a8e175 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:32.068759 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a537a7-c0d5-4ce3-971e-c7e583a8e175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:32.079022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a537a7-c0d5-4ce3-971e-c7e583a8e175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:32.080473 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a537a7-c0d5-4ce3-971e-c7e583a8e175 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:32.089994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1029/2062] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:39:32.355575 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa1366-032d-46b6-be3f-f8ca2483b179 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:32.356797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-aeaa1366-032d-46b6-be3f-f8ca2483b179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:32.369934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa1366-032d-46b6-be3f-f8ca2483b179 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:32.376452 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039df142-30dc-4191-aa63-826558402393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:32.377327 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039df142-30dc-4191-aa63-826558402393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:32.377612 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039df142-30dc-4191-aa63-826558402393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:32.377889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039df142-30dc-4191-aa63-826558402393 admin admin] PATH_INFO: `/v3/projects/056fbc9190d54341bc2a92f3c608f409` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:32.379085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039df142-30dc-4191-aa63-826558402393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=056fbc9190d54341bc2a92f3c608f409)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:32.381996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039df142-30dc-4191-aa63-826558402393 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:32.435419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:32.436773 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-039df142-30dc-4191-aa63-826558402393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '056fbc9190d54341bc2a92f3c608f409', 'request_id': 'req-039df142-30dc-4191-aa63-826558402393'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:32.444201 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-039df142-30dc-4191-aa63-826558402393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '056fbc9190d54341bc2a92f3c608f409', 'request_id': 'req-039df142-30dc-4191-aa63-826558402393'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:32.450615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-039df142-30dc-4191-aa63-826558402393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '056fbc9190d54341bc2a92f3c608f409', 'request_id': 'req-039df142-30dc-4191-aa63-826558402393'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:32.461412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-039df142-30dc-4191-aa63-826558402393 admin admin] The token cache is being invalidate because project 056fbc9190d54341bc2a92f3c608f409 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:39:32.461702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-039df142-30dc-4191-aa63-826558402393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:32.462327 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-039df142-30dc-4191-aa63-826558402393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:32.463960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1034/2063] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:39:32 2022] DELETE /identity/v3/projects/056fbc9190d54341bc2a92f3c608f409 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:32.468094 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a537a7-c0d5-4ce3-971e-c7e583a8e175 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:32.477513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a537a7-c0d5-4ce3-971e-c7e583a8e175 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:32.534646 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5d7dd-aed0-4707-ae70-53d80a56013e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:32.535427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5d7dd-aed0-4707-ae70-53d80a56013e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:32.535738 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5d7dd-aed0-4707-ae70-53d80a56013e admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:32.536037 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5d7dd-aed0-4707-ae70-53d80a56013e admin admin] PATH_INFO: `/v3/users/575d93a5ba78440c86e98d2a863ac372` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:32.537194 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a5d7dd-aed0-4707-ae70-53d80a56013e admin admin] RBAC: Authorizing `identity:delete_user(user_id=575d93a5ba78440c86e98d2a863ac372)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:32.539077 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a5d7dd-aed0-4707-ae70-53d80a56013e admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:32.587772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-36a5d7dd-aed0-4707-ae70-53d80a56013e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '575d93a5ba78440c86e98d2a863ac372'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:32.594589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-36a5d7dd-aed0-4707-ae70-53d80a56013e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '575d93a5ba78440c86e98d2a863ac372'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:32.602124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-36a5d7dd-aed0-4707-ae70-53d80a56013e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '575d93a5ba78440c86e98d2a863ac372'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:32.602920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-36a5d7dd-aed0-4707-ae70-53d80a56013e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '575d93a5ba78440c86e98d2a863ac372'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:32.614456 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-36a5d7dd-aed0-4707-ae70-53d80a56013e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '575d93a5ba78440c86e98d2a863ac372'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:32.621260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1030/2064] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:39:32 2022] DELETE /identity/v3/users/575d93a5ba78440c86e98d2a863ac372 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:32.966844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039df142-30dc-4191-aa63-826558402393 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:32.968990 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-039df142-30dc-4191-aa63-826558402393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:32.989181 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039df142-30dc-4191-aa63-826558402393 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:33.047490 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d064c-f47b-47be-85b1-afebbc26ffe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:33.048407 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d064c-f47b-47be-85b1-afebbc26ffe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:33.048690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d064c-f47b-47be-85b1-afebbc26ffe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:33.048964 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d064c-f47b-47be-85b1-afebbc26ffe6 admin admin] PATH_INFO: `/v3/projects/ba81c02d82514a3bb2a4388d1a1d3344` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:33.050215 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d064c-f47b-47be-85b1-afebbc26ffe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba81c02d82514a3bb2a4388d1a1d3344)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:33.052353 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d064c-f47b-47be-85b1-afebbc26ffe6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:33.099192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-394d064c-f47b-47be-85b1-afebbc26ffe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba81c02d82514a3bb2a4388d1a1d3344', 'request_id': 'req-394d064c-f47b-47be-85b1-afebbc26ffe6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:33.107139 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-394d064c-f47b-47be-85b1-afebbc26ffe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba81c02d82514a3bb2a4388d1a1d3344', 'request_id': 'req-394d064c-f47b-47be-85b1-afebbc26ffe6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:33.112673 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-394d064c-f47b-47be-85b1-afebbc26ffe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba81c02d82514a3bb2a4388d1a1d3344', 'request_id': 'req-394d064c-f47b-47be-85b1-afebbc26ffe6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:33.123990 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-394d064c-f47b-47be-85b1-afebbc26ffe6 admin admin] The token cache is being invalidate because project ba81c02d82514a3bb2a4388d1a1d3344 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:39:33.124267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-394d064c-f47b-47be-85b1-afebbc26ffe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:33.124885 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-394d064c-f47b-47be-85b1-afebbc26ffe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:33.126406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1035/2065] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:39:32 2022] DELETE /identity/v3/projects/ba81c02d82514a3bb2a4388d1a1d3344 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:33.131151 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ba4da-678d-4c50-99d2-9b35287b0769 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:33.131606 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ba4da-678d-4c50-99d2-9b35287b0769 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:33.131695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ba4da-678d-4c50-99d2-9b35287b0769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:33.149078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-990ba4da-678d-4c50-99d2-9b35287b0769 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:33.150257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-990ba4da-678d-4c50-99d2-9b35287b0769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:33.185407 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:33.213572 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1031/2066] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:33.221227 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d81f3d-75a3-4953-afa9-2d2fed2e8868 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:33.221514 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d81f3d-75a3-4953-afa9-2d2fed2e8868 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:33.221789 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d81f3d-75a3-4953-afa9-2d2fed2e8868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:33.240734 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-87d81f3d-75a3-4953-afa9-2d2fed2e8868 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:33.241953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-87d81f3d-75a3-4953-afa9-2d2fed2e8868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:33.250857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1036/2067] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:33.255542 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ba4da-678d-4c50-99d2-9b35287b0769 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:33.268257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ba4da-678d-4c50-99d2-9b35287b0769 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:33.274602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612710d8-0f85-4009-8158-0d43d1b271ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:33.275271 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612710d8-0f85-4009-8158-0d43d1b271ad admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:33.275555 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612710d8-0f85-4009-8158-0d43d1b271ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:33.275858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612710d8-0f85-4009-8158-0d43d1b271ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:33.277865 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612710d8-0f85-4009-8158-0d43d1b271ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:39:33.278179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612710d8-0f85-4009-8158-0d43d1b271ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:33.282371 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612710d8-0f85-4009-8158-0d43d1b271ad admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:33.298461 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1032/2068] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:39:33.302424 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d81f3d-75a3-4953-afa9-2d2fed2e8868 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:33.311833 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d81f3d-75a3-4953-afa9-2d2fed2e8868 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:33.316583 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a4fd7-36e6-4f42-92e6-41e2cafe6ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:33.317243 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a4fd7-36e6-4f42-92e6-41e2cafe6ddb admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:33.317580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a4fd7-36e6-4f42-92e6-41e2cafe6ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:33.317879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a4fd7-36e6-4f42-92e6-41e2cafe6ddb admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:33.318630 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a4fd7-36e6-4f42-92e6-41e2cafe6ddb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:33.320580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a4fd7-36e6-4f42-92e6-41e2cafe6ddb admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:33.359303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1037/2069] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:39:33 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:39:33.360707 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612710d8-0f85-4009-8158-0d43d1b271ad admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:33.371442 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612710d8-0f85-4009-8158-0d43d1b271ad admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:33.431167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c90cb7-a101-4f29-bcf9-b4f635777d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:33.431846 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c90cb7-a101-4f29-bcf9-b4f635777d9d admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:33.432128 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c90cb7-a101-4f29-bcf9-b4f635777d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:33.432399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c90cb7-a101-4f29-bcf9-b4f635777d9d admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:33.433512 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c90cb7-a101-4f29-bcf9-b4f635777d9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:33.435403 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c90cb7-a101-4f29-bcf9-b4f635777d9d admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:33.454781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1033/2070] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:39:33 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:33.458982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a4fd7-36e6-4f42-92e6-41e2cafe6ddb admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:33.471990 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a4fd7-36e6-4f42-92e6-41e2cafe6ddb admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:33.477297 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ae117-9101-40cf-ab19-9f8c47a178ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:33.477948 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ae117-9101-40cf-ab19-9f8c47a178ae admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:33.478232 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ae117-9101-40cf-ab19-9f8c47a178ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:33.478506 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ae117-9101-40cf-ab19-9f8c47a178ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:33.479228 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ae117-9101-40cf-ab19-9f8c47a178ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:33.481698 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ae117-9101-40cf-ab19-9f8c47a178ae admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:33.491922 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1038/2071] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:33.495939 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c90cb7-a101-4f29-bcf9-b4f635777d9d admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:33.505721 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c90cb7-a101-4f29-bcf9-b4f635777d9d admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:33.510294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275bf5f-a1fc-48b5-a7db-12ef232e78ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:33.510974 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275bf5f-a1fc-48b5-a7db-12ef232e78ad admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:33.511243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275bf5f-a1fc-48b5-a7db-12ef232e78ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:33.511518 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275bf5f-a1fc-48b5-a7db-12ef232e78ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:33.512260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275bf5f-a1fc-48b5-a7db-12ef232e78ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:33.513834 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275bf5f-a1fc-48b5-a7db-12ef232e78ad admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:33.524171 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1034/2072] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:33.528750 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ae117-9101-40cf-ab19-9f8c47a178ae admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:33.538454 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ae117-9101-40cf-ab19-9f8c47a178ae admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:33.543578 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd041781-fd0e-4d6e-b653-85f3bcc8aaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:33.544478 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd041781-fd0e-4d6e-b653-85f3bcc8aaf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:33.544759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd041781-fd0e-4d6e-b653-85f3bcc8aaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:33.545029 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd041781-fd0e-4d6e-b653-85f3bcc8aaf8 admin admin] PATH_INFO: `/v3/projects/ee48ac94a2414c70963263a637921502/users/061ea711fcb9428386e6bf7344146ae5/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:33.559198 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd041781-fd0e-4d6e-b653-85f3bcc8aaf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee48ac94a2414c70963263a637921502, user_id=061ea711fcb9428386e6bf7344146ae5, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:33.562061 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd041781-fd0e-4d6e-b653-85f3bcc8aaf8 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:33.574656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1039/2073] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:33 2022] PUT /identity/v3/projects/ee48ac94a2414c70963263a637921502/users/061ea711fcb9428386e6bf7344146ae5/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:39:33.585146 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e80178-7e8e-4b87-b606-7ae9d49cba01 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:33.585474 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e80178-7e8e-4b87-b606-7ae9d49cba01 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:33.585761 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e80178-7e8e-4b87-b606-7ae9d49cba01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:33.614170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-f5e80178-7e8e-4b87-b606-7ae9d49cba01 None None] MFA Rules not processed for user `061ea711fcb9428386e6bf7344146ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:33.615386 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-f5e80178-7e8e-4b87-b606-7ae9d49cba01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:33.680492 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1035/2074] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:33.684974 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd041781-fd0e-4d6e-b653-85f3bcc8aaf8 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:33.689354 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:33.751107 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd041781-fd0e-4d6e-b653-85f3bcc8aaf8 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:33.755823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e0aca-2355-4f9e-a61c-5af10cc6a293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:33.756474 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e0aca-2355-4f9e-a61c-5af10cc6a293 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:33.756762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e0aca-2355-4f9e-a61c-5af10cc6a293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:33.757042 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e0aca-2355-4f9e-a61c-5af10cc6a293 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:33.757734 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e0aca-2355-4f9e-a61c-5af10cc6a293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:33.759638 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e0aca-2355-4f9e-a61c-5af10cc6a293 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:33.786620 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1040/2075] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:39:33 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:33.790818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e80178-7e8e-4b87-b606-7ae9d49cba01 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:33.801697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e80178-7e8e-4b87-b606-7ae9d49cba01 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:33.833595 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:33.859351 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230e283-0f6f-4aa4-8f51-5fb224743322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:33.860044 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230e283-0f6f-4aa4-8f51-5fb224743322 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:33.860326 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230e283-0f6f-4aa4-8f51-5fb224743322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:33.860603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230e283-0f6f-4aa4-8f51-5fb224743322 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:33.861278 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230e283-0f6f-4aa4-8f51-5fb224743322 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:33.863173 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230e283-0f6f-4aa4-8f51-5fb224743322 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:33.882443 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1036/2076] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:39:33 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:39:33.886923 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e0aca-2355-4f9e-a61c-5af10cc6a293 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:33.897276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e0aca-2355-4f9e-a61c-5af10cc6a293 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:33.902385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8918c0-14c4-4611-a248-b82a7a33d541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:33.903029 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8918c0-14c4-4611-a248-b82a7a33d541 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:33.903312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8918c0-14c4-4611-a248-b82a7a33d541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:33.903586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8918c0-14c4-4611-a248-b82a7a33d541 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:33.904346 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8918c0-14c4-4611-a248-b82a7a33d541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:33.905949 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8918c0-14c4-4611-a248-b82a7a33d541 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:33.916289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1041/2077] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:33.920365 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230e283-0f6f-4aa4-8f51-5fb224743322 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:33.930356 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230e283-0f6f-4aa4-8f51-5fb224743322 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:33.935102 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c55867-e526-4f39-8974-12f120ff828b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:33.936016 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c55867-e526-4f39-8974-12f120ff828b admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:33.936303 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c55867-e526-4f39-8974-12f120ff828b admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:33.936581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c55867-e526-4f39-8974-12f120ff828b admin admin] PATH_INFO: `/v3/projects/5d7a045b3bfa4af18bf51c8d525f2758/users/6c10e4eeddbc43bba2fdcd377b0dbb5f/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:33.952995 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c55867-e526-4f39-8974-12f120ff828b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d7a045b3bfa4af18bf51c8d525f2758, user_id=6c10e4eeddbc43bba2fdcd377b0dbb5f, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:33.955734 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c55867-e526-4f39-8974-12f120ff828b admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:33.969163 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1037/2078] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:33 2022] PUT /identity/v3/projects/5d7a045b3bfa4af18bf51c8d525f2758/users/6c10e4eeddbc43bba2fdcd377b0dbb5f/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:33.973137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8918c0-14c4-4611-a248-b82a7a33d541 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:33.983958 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8918c0-14c4-4611-a248-b82a7a33d541 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:34.045168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4d291-637d-4ecb-aa49-bdb5ed71489d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:34.045841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4d291-637d-4ecb-aa49-bdb5ed71489d admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:34.046123 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4d291-637d-4ecb-aa49-bdb5ed71489d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:34.046395 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4d291-637d-4ecb-aa49-bdb5ed71489d admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:34.047134 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4d291-637d-4ecb-aa49-bdb5ed71489d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:34.048741 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4d291-637d-4ecb-aa49-bdb5ed71489d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:34.059719 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1042/2079] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:34.064177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c55867-e526-4f39-8974-12f120ff828b admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:34.075601 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c55867-e526-4f39-8974-12f120ff828b admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:34.081929 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cc4aa-1f99-4568-83b5-1b52c24aa635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:34.082893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cc4aa-1f99-4568-83b5-1b52c24aa635 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:34.083216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cc4aa-1f99-4568-83b5-1b52c24aa635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:34.083550 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cc4aa-1f99-4568-83b5-1b52c24aa635 admin admin] PATH_INFO: `/v3/projects/5d7a045b3bfa4af18bf51c8d525f2758/users/6c10e4eeddbc43bba2fdcd377b0dbb5f/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:34.085742 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718cc4aa-1f99-4568-83b5-1b52c24aa635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d7a045b3bfa4af18bf51c8d525f2758, user_id=6c10e4eeddbc43bba2fdcd377b0dbb5f, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:34.089300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718cc4aa-1f99-4568-83b5-1b52c24aa635 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:34.101788 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1038/2080] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:34 2022] PUT /identity/v3/projects/5d7a045b3bfa4af18bf51c8d525f2758/users/6c10e4eeddbc43bba2fdcd377b0dbb5f/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:34.114725 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a53cd0-4613-46bb-8b85-36da60107682 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:34.115036 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a53cd0-4613-46bb-8b85-36da60107682 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:34.115321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a53cd0-4613-46bb-8b85-36da60107682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:34.160387 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-f5a53cd0-4613-46bb-8b85-36da60107682 None None] MFA Rules not processed for user `6c10e4eeddbc43bba2fdcd377b0dbb5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:34.162146 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f5a53cd0-4613-46bb-8b85-36da60107682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:34.234821 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:34.246085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1043/2081] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:39:34.256459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cc4aa-1f99-4568-83b5-1b52c24aa635 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:34.258437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-718cc4aa-1f99-4568-83b5-1b52c24aa635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:34.327284 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cc4aa-1f99-4568-83b5-1b52c24aa635 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:34.331328 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd1219-58d8-4ca7-92a1-fc5020c2d8c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:34.332113 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd1219-58d8-4ca7-92a1-fc5020c2d8c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:34.332468 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd1219-58d8-4ca7-92a1-fc5020c2d8c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:34.332789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd1219-58d8-4ca7-92a1-fc5020c2d8c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:34.410498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd1219-58d8-4ca7-92a1-fc5020c2d8c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:34.411977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd1219-58d8-4ca7-92a1-fc5020c2d8c1 service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:34.422658 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1039/2082] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:39:35.185246 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a53cd0-4613-46bb-8b85-36da60107682 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:35.186767 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f5a53cd0-4613-46bb-8b85-36da60107682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:35.253206 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a53cd0-4613-46bb-8b85-36da60107682 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:35.258005 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f53b73-955d-4eb7-9ac4-9efaf399c749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:35.258877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f53b73-955d-4eb7-9ac4-9efaf399c749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:35.259156 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f53b73-955d-4eb7-9ac4-9efaf399c749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:35.259456 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f53b73-955d-4eb7-9ac4-9efaf399c749 admin admin] PATH_INFO: `/v3/users/112bd2b1bae64be699cfe0e5a5aede52` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:35.261240 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f53b73-955d-4eb7-9ac4-9efaf399c749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=112bd2b1bae64be699cfe0e5a5aede52)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:35.263156 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f53b73-955d-4eb7-9ac4-9efaf399c749 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:35.312637 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c1f53b73-955d-4eb7-9ac4-9efaf399c749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '112bd2b1bae64be699cfe0e5a5aede52'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:35.319464 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c1f53b73-955d-4eb7-9ac4-9efaf399c749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '112bd2b1bae64be699cfe0e5a5aede52'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:35.320144 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c1f53b73-955d-4eb7-9ac4-9efaf399c749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '112bd2b1bae64be699cfe0e5a5aede52'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:35.320756 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c1f53b73-955d-4eb7-9ac4-9efaf399c749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '112bd2b1bae64be699cfe0e5a5aede52'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:35.333679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c1f53b73-955d-4eb7-9ac4-9efaf399c749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '112bd2b1bae64be699cfe0e5a5aede52'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:35.347378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1044/2083] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:39:35 2022] DELETE /identity/v3/users/112bd2b1bae64be699cfe0e5a5aede52 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:35.351987 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fe096-7b9e-44a4-8516-8a7a0bc907bb None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:35.352377 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fe096-7b9e-44a4-8516-8a7a0bc907bb None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:35.352738 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fe096-7b9e-44a4-8516-8a7a0bc907bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:35.371081 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-572fe096-7b9e-44a4-8516-8a7a0bc907bb None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:35.372336 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-572fe096-7b9e-44a4-8516-8a7a0bc907bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:35.416027 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:35.439991 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1040/2084] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:35.450632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f53b73-955d-4eb7-9ac4-9efaf399c749 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:35.452187 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c1f53b73-955d-4eb7-9ac4-9efaf399c749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:35.515004 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f53b73-955d-4eb7-9ac4-9efaf399c749 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:35.519211 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea71cea-0d3d-4130-8011-dacf170edef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:35.520199 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea71cea-0d3d-4130-8011-dacf170edef2 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:35.520493 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea71cea-0d3d-4130-8011-dacf170edef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:35.520808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea71cea-0d3d-4130-8011-dacf170edef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:35.532383 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea71cea-0d3d-4130-8011-dacf170edef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:35.533794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea71cea-0d3d-4130-8011-dacf170edef2 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:35.544390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1045/2085] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:39:35.956272 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fe096-7b9e-44a4-8516-8a7a0bc907bb None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:35.959268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-572fe096-7b9e-44a4-8516-8a7a0bc907bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:35.981944 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fe096-7b9e-44a4-8516-8a7a0bc907bb None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:36.005498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e53dc-60a2-49d6-a3aa-a093f833d702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:36.006446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e53dc-60a2-49d6-a3aa-a093f833d702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:36.006800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e53dc-60a2-49d6-a3aa-a093f833d702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:36.008782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e53dc-60a2-49d6-a3aa-a093f833d702 admin admin] PATH_INFO: `/v3/projects/69001a078435420ca20c70e406c217bf` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:36.010138 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e53dc-60a2-49d6-a3aa-a093f833d702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69001a078435420ca20c70e406c217bf)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:36.015303 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e53dc-60a2-49d6-a3aa-a093f833d702 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:36.113019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-024e53dc-60a2-49d6-a3aa-a093f833d702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69001a078435420ca20c70e406c217bf', 'request_id': 'req-024e53dc-60a2-49d6-a3aa-a093f833d702'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:36.118840 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-024e53dc-60a2-49d6-a3aa-a093f833d702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69001a078435420ca20c70e406c217bf', 'request_id': 'req-024e53dc-60a2-49d6-a3aa-a093f833d702'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:36.127299 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-024e53dc-60a2-49d6-a3aa-a093f833d702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69001a078435420ca20c70e406c217bf', 'request_id': 'req-024e53dc-60a2-49d6-a3aa-a093f833d702'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:36.140002 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-024e53dc-60a2-49d6-a3aa-a093f833d702 admin admin] The token cache is being invalidate because project 69001a078435420ca20c70e406c217bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:39:36.140437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-024e53dc-60a2-49d6-a3aa-a093f833d702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:36.141069 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-024e53dc-60a2-49d6-a3aa-a093f833d702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:36.142595 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1041/2086] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:39:35 2022] DELETE /identity/v3/projects/69001a078435420ca20c70e406c217bf => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:36.152609 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19069a29-31da-428c-bbe3-9da778568d3e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:36.152928 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19069a29-31da-428c-bbe3-9da778568d3e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:36.153209 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19069a29-31da-428c-bbe3-9da778568d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:36.171061 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-19069a29-31da-428c-bbe3-9da778568d3e None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:36.172278 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-19069a29-31da-428c-bbe3-9da778568d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:36.185596 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:36.233098 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1046/2087] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:36.241313 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e1fbd-3838-4b09-ac47-f8a321990896 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:36.241624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e1fbd-3838-4b09-ac47-f8a321990896 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:36.241903 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e1fbd-3838-4b09-ac47-f8a321990896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:36.261364 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-355e1fbd-3838-4b09-ac47-f8a321990896 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:36.262562 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-355e1fbd-3838-4b09-ac47-f8a321990896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:36.270699 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1042/2088] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:39:36.275006 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19069a29-31da-428c-bbe3-9da778568d3e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:36.286313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19069a29-31da-428c-bbe3-9da778568d3e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:36.291090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2b61e-4f18-443e-af39-6057d350b4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:36.291837 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2b61e-4f18-443e-af39-6057d350b4a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:36.292715 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2b61e-4f18-443e-af39-6057d350b4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:36.292994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2b61e-4f18-443e-af39-6057d350b4a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:36.293716 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2b61e-4f18-443e-af39-6057d350b4a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:39:36.294025 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2b61e-4f18-443e-af39-6057d350b4a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:36.295770 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2b61e-4f18-443e-af39-6057d350b4a5 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:36.307892 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1047/2089] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:39:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:39:36.312498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e1fbd-3838-4b09-ac47-f8a321990896 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:36.321730 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e1fbd-3838-4b09-ac47-f8a321990896 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:36.327094 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed905772-d4c5-4df2-9bf6-3cdbaee38576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:36.327812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed905772-d4c5-4df2-9bf6-3cdbaee38576 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:36.328115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed905772-d4c5-4df2-9bf6-3cdbaee38576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:36.328388 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed905772-d4c5-4df2-9bf6-3cdbaee38576 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:36.329070 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed905772-d4c5-4df2-9bf6-3cdbaee38576 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:36.331089 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed905772-d4c5-4df2-9bf6-3cdbaee38576 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:36.363916 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1043/2090] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:39:36 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:36.365938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2b61e-4f18-443e-af39-6057d350b4a5 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:36.379035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2b61e-4f18-443e-af39-6057d350b4a5 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:36.440362 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45dbcc0-565d-4bc5-829a-a6cec35314db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:36.440942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45dbcc0-565d-4bc5-829a-a6cec35314db admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:36.441228 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45dbcc0-565d-4bc5-829a-a6cec35314db admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:36.441504 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45dbcc0-565d-4bc5-829a-a6cec35314db admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:36.442203 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45dbcc0-565d-4bc5-829a-a6cec35314db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:36.444143 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45dbcc0-565d-4bc5-829a-a6cec35314db admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:36.465930 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1048/2091] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:39:36 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:39:36.470139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed905772-d4c5-4df2-9bf6-3cdbaee38576 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:36.480697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed905772-d4c5-4df2-9bf6-3cdbaee38576 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:36.485319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4db6ff-2345-4917-a464-bc4f7ec77363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:36.485978 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4db6ff-2345-4917-a464-bc4f7ec77363 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:36.486261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4db6ff-2345-4917-a464-bc4f7ec77363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:36.486540 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4db6ff-2345-4917-a464-bc4f7ec77363 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:36.487461 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4db6ff-2345-4917-a464-bc4f7ec77363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:36.488918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4db6ff-2345-4917-a464-bc4f7ec77363 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:36.500368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1044/2092] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:36.504321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45dbcc0-565d-4bc5-829a-a6cec35314db admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:36.515694 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45dbcc0-565d-4bc5-829a-a6cec35314db admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:36.520832 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9a855-0ca6-46e5-8c18-64e7c5f93c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:36.521626 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9a855-0ca6-46e5-8c18-64e7c5f93c8e admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:36.521989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9a855-0ca6-46e5-8c18-64e7c5f93c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:36.522268 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9a855-0ca6-46e5-8c18-64e7c5f93c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:36.522996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9a855-0ca6-46e5-8c18-64e7c5f93c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:36.524618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9a855-0ca6-46e5-8c18-64e7c5f93c8e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:36.535125 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1049/2093] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:36.539441 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4db6ff-2345-4917-a464-bc4f7ec77363 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:36.549863 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4db6ff-2345-4917-a464-bc4f7ec77363 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:36.554558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f795e4b-51a7-4618-8515-1d1e0329e453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:36.555438 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f795e4b-51a7-4618-8515-1d1e0329e453 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:36.555723 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f795e4b-51a7-4618-8515-1d1e0329e453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:36.556029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f795e4b-51a7-4618-8515-1d1e0329e453 admin admin] PATH_INFO: `/v3/projects/8f194c7715ce4cbb85c1930f506e6138/users/fb7ff822974947c893a2137c28f8df9c/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:36.571463 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f795e4b-51a7-4618-8515-1d1e0329e453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f194c7715ce4cbb85c1930f506e6138, user_id=fb7ff822974947c893a2137c28f8df9c, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:36.574815 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f795e4b-51a7-4618-8515-1d1e0329e453 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:36.587574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1045/2094] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:36 2022] PUT /identity/v3/projects/8f194c7715ce4cbb85c1930f506e6138/users/fb7ff822974947c893a2137c28f8df9c/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:36.599073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445705c9-ac0b-4f81-9d39-edaa69652522 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:36.599395 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445705c9-ac0b-4f81-9d39-edaa69652522 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:36.599679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445705c9-ac0b-4f81-9d39-edaa69652522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:36.626047 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-445705c9-ac0b-4f81-9d39-edaa69652522 None None] MFA Rules not processed for user `fb7ff822974947c893a2137c28f8df9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:36.627245 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-445705c9-ac0b-4f81-9d39-edaa69652522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:36.690485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1050/2095] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:36 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:36.695002 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f795e4b-51a7-4618-8515-1d1e0329e453 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:36.756166 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f795e4b-51a7-4618-8515-1d1e0329e453 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:36.760808 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46438638-d6a8-4414-a543-1628b114d2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:36.761442 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46438638-d6a8-4414-a543-1628b114d2e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:36.761716 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46438638-d6a8-4414-a543-1628b114d2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:36.761985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46438638-d6a8-4414-a543-1628b114d2e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:36.762666 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46438638-d6a8-4414-a543-1628b114d2e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:36.764559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46438638-d6a8-4414-a543-1628b114d2e0 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:36.768466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:36.791061 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1046/2096] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:39:36 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:36.795550 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445705c9-ac0b-4f81-9d39-edaa69652522 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:36.806218 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445705c9-ac0b-4f81-9d39-edaa69652522 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:36.818461 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:36.871808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5858c-b312-4c39-9a45-0b549daabc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:36.872518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5858c-b312-4c39-9a45-0b549daabc62 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:36.872775 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5858c-b312-4c39-9a45-0b549daabc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:36.873043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5858c-b312-4c39-9a45-0b549daabc62 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:36.873734 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5858c-b312-4c39-9a45-0b549daabc62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:36.875639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5858c-b312-4c39-9a45-0b549daabc62 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:36.895795 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1051/2097] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:39:36 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:36.899695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46438638-d6a8-4414-a543-1628b114d2e0 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:36.910894 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46438638-d6a8-4414-a543-1628b114d2e0 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:36.915972 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce4ffa-cbfc-4dac-b8db-3fce3064889f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:36.917213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce4ffa-cbfc-4dac-b8db-3fce3064889f admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:36.917639 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce4ffa-cbfc-4dac-b8db-3fce3064889f admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:36.917987 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce4ffa-cbfc-4dac-b8db-3fce3064889f admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:36.918732 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce4ffa-cbfc-4dac-b8db-3fce3064889f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:36.920354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce4ffa-cbfc-4dac-b8db-3fce3064889f admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:36.931129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1047/2098] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:36.935942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5858c-b312-4c39-9a45-0b549daabc62 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:36.946699 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5858c-b312-4c39-9a45-0b549daabc62 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:36.951442 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9049d1-93ce-4686-aaa4-6faaf98cfac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:36.952692 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9049d1-93ce-4686-aaa4-6faaf98cfac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:36.953447 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9049d1-93ce-4686-aaa4-6faaf98cfac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:36.953447 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9049d1-93ce-4686-aaa4-6faaf98cfac3 admin admin] PATH_INFO: `/v3/projects/eca6d0c2ad714890909e1fd7426aac6e/users/eced5f58c74d41aabe3e7e48cfbc5f52/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:36.973989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9049d1-93ce-4686-aaa4-6faaf98cfac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eca6d0c2ad714890909e1fd7426aac6e, user_id=eced5f58c74d41aabe3e7e48cfbc5f52, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:36.977372 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9049d1-93ce-4686-aaa4-6faaf98cfac3 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:36.993291 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1052/2099] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:36 2022] PUT /identity/v3/projects/eca6d0c2ad714890909e1fd7426aac6e/users/eced5f58c74d41aabe3e7e48cfbc5f52/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:36.998059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce4ffa-cbfc-4dac-b8db-3fce3064889f admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:37.008335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce4ffa-cbfc-4dac-b8db-3fce3064889f admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:37.066196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a92753-d0f5-498f-9ffc-2042ae328c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:37.066882 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a92753-d0f5-498f-9ffc-2042ae328c7f admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:37.067167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a92753-d0f5-498f-9ffc-2042ae328c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:37.067436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a92753-d0f5-498f-9ffc-2042ae328c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:37.068183 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a92753-d0f5-498f-9ffc-2042ae328c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:37.069761 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a92753-d0f5-498f-9ffc-2042ae328c7f admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:37.080285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1048/2100] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:39:37.084611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9049d1-93ce-4686-aaa4-6faaf98cfac3 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:37.101726 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9049d1-93ce-4686-aaa4-6faaf98cfac3 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:37.107725 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb7dab-2c4f-4049-907c-48cd72e22700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:37.108877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb7dab-2c4f-4049-907c-48cd72e22700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:37.108971 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb7dab-2c4f-4049-907c-48cd72e22700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:37.109212 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb7dab-2c4f-4049-907c-48cd72e22700 admin admin] PATH_INFO: `/v3/projects/eca6d0c2ad714890909e1fd7426aac6e/users/eced5f58c74d41aabe3e7e48cfbc5f52/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:37.111200 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb7dab-2c4f-4049-907c-48cd72e22700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eca6d0c2ad714890909e1fd7426aac6e, user_id=eced5f58c74d41aabe3e7e48cfbc5f52, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:37.113934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb7dab-2c4f-4049-907c-48cd72e22700 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:37.124614 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1053/2101] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:37 2022] PUT /identity/v3/projects/eca6d0c2ad714890909e1fd7426aac6e/users/eced5f58c74d41aabe3e7e48cfbc5f52/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:37.134763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e42b5-1c9f-4b84-89f3-3eb53cf9dc00 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:37.135050 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e42b5-1c9f-4b84-89f3-3eb53cf9dc00 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:37.135325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e42b5-1c9f-4b84-89f3-3eb53cf9dc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:37.161755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-3c4e42b5-1c9f-4b84-89f3-3eb53cf9dc00 None None] MFA Rules not processed for user `eced5f58c74d41aabe3e7e48cfbc5f52`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:37.162935 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-3c4e42b5-1c9f-4b84-89f3-3eb53cf9dc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:37.234049 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1049/2102] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:37 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:37.244959 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb7dab-2c4f-4049-907c-48cd72e22700 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:37.246768 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-14fb7dab-2c4f-4049-907c-48cd72e22700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:37.309310 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb7dab-2c4f-4049-907c-48cd72e22700 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:37.313014 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd2a32-8808-44ab-afe3-11c7242bedd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:37.313645 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd2a32-8808-44ab-afe3-11c7242bedd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:37.313924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd2a32-8808-44ab-afe3-11c7242bedd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:37.314195 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd2a32-8808-44ab-afe3-11c7242bedd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:37.322957 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd2a32-8808-44ab-afe3-11c7242bedd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:37.324421 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd2a32-8808-44ab-afe3-11c7242bedd9 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:37.333501 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1054/2103] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:37 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:39:37.710309 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e42b5-1c9f-4b84-89f3-3eb53cf9dc00 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:37.768881 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:37.773468 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e42b5-1c9f-4b84-89f3-3eb53cf9dc00 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:37.779571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7b4ee-3e6d-4573-b26e-fb5b09307d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:37.780540 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7b4ee-3e6d-4573-b26e-fb5b09307d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:37.780817 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7b4ee-3e6d-4573-b26e-fb5b09307d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:37.781088 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7b4ee-3e6d-4573-b26e-fb5b09307d04 admin admin] PATH_INFO: `/v3/users/fb7ff822974947c893a2137c28f8df9c` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:37.782243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7b4ee-3e6d-4573-b26e-fb5b09307d04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb7ff822974947c893a2137c28f8df9c)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:37.784202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7b4ee-3e6d-4573-b26e-fb5b09307d04 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:37.830193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3de7b4ee-3e6d-4573-b26e-fb5b09307d04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb7ff822974947c893a2137c28f8df9c'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:37.836830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3de7b4ee-3e6d-4573-b26e-fb5b09307d04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb7ff822974947c893a2137c28f8df9c'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:37.844367 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3de7b4ee-3e6d-4573-b26e-fb5b09307d04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb7ff822974947c893a2137c28f8df9c'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:37.844976 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3de7b4ee-3e6d-4573-b26e-fb5b09307d04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb7ff822974947c893a2137c28f8df9c'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:37.857255 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3de7b4ee-3e6d-4573-b26e-fb5b09307d04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb7ff822974947c893a2137c28f8df9c'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:37.863056 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1050/2104] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:39:37 2022] DELETE /identity/v3/users/fb7ff822974947c893a2137c28f8df9c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:37.867686 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1e899-50ff-40f6-b834-b7c97d1297dc None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:37.868064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1e899-50ff-40f6-b834-b7c97d1297dc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:37.868493 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1e899-50ff-40f6-b834-b7c97d1297dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:37.887948 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-4cf1e899-50ff-40f6-b834-b7c97d1297dc None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:37.889130 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4cf1e899-50ff-40f6-b834-b7c97d1297dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:37.947021 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1055/2105] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:37.957179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7b4ee-3e6d-4573-b26e-fb5b09307d04 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:37.958629 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-3de7b4ee-3e6d-4573-b26e-fb5b09307d04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:38.016164 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7b4ee-3e6d-4573-b26e-fb5b09307d04 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:38.020338 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e60317-666e-4223-b98e-7749982b63e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:38.020916 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e60317-666e-4223-b98e-7749982b63e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:38.021199 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e60317-666e-4223-b98e-7749982b63e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:38.021506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e60317-666e-4223-b98e-7749982b63e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:38.031161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e60317-666e-4223-b98e-7749982b63e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:38.032619 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e60317-666e-4223-b98e-7749982b63e3 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:38.042523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1051/2106] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:39:38.415082 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1e899-50ff-40f6-b834-b7c97d1297dc None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:38.416528 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4cf1e899-50ff-40f6-b834-b7c97d1297dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:38.431363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1e899-50ff-40f6-b834-b7c97d1297dc None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:38.437820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bf3e0-21ad-460c-aa95-93b7debefa34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:38.438699 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bf3e0-21ad-460c-aa95-93b7debefa34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:38.438977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bf3e0-21ad-460c-aa95-93b7debefa34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:38.439249 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bf3e0-21ad-460c-aa95-93b7debefa34 admin admin] PATH_INFO: `/v3/projects/8f194c7715ce4cbb85c1930f506e6138` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:38.440463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27bf3e0-21ad-460c-aa95-93b7debefa34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f194c7715ce4cbb85c1930f506e6138)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:38.442341 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27bf3e0-21ad-460c-aa95-93b7debefa34 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:38.448890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:38.489695 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d27bf3e0-21ad-460c-aa95-93b7debefa34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f194c7715ce4cbb85c1930f506e6138', 'request_id': 'req-d27bf3e0-21ad-460c-aa95-93b7debefa34'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:38.497430 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d27bf3e0-21ad-460c-aa95-93b7debefa34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f194c7715ce4cbb85c1930f506e6138', 'request_id': 'req-d27bf3e0-21ad-460c-aa95-93b7debefa34'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:38.503721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d27bf3e0-21ad-460c-aa95-93b7debefa34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f194c7715ce4cbb85c1930f506e6138', 'request_id': 'req-d27bf3e0-21ad-460c-aa95-93b7debefa34'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:38.514924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d27bf3e0-21ad-460c-aa95-93b7debefa34 admin admin] The token cache is being invalidate because project 8f194c7715ce4cbb85c1930f506e6138 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:39:38.515210 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d27bf3e0-21ad-460c-aa95-93b7debefa34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:38.516149 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d27bf3e0-21ad-460c-aa95-93b7debefa34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:38.517726 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1056/2107] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:39:38 2022] DELETE /identity/v3/projects/8f194c7715ce4cbb85c1930f506e6138 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:38.521937 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e60317-666e-4223-b98e-7749982b63e3 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:38.532138 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e60317-666e-4223-b98e-7749982b63e3 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:38.586723 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c1545-08d9-4952-8bf7-8745d5605587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:38.588026 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c1545-08d9-4952-8bf7-8745d5605587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:38.589067 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c1545-08d9-4952-8bf7-8745d5605587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:38.589524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c1545-08d9-4952-8bf7-8745d5605587 admin admin] PATH_INFO: `/v3/users/eced5f58c74d41aabe3e7e48cfbc5f52` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:38.590736 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c1545-08d9-4952-8bf7-8745d5605587 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eced5f58c74d41aabe3e7e48cfbc5f52)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:38.593131 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c1545-08d9-4952-8bf7-8745d5605587 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:38.617643 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:38.641461 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d85c1545-08d9-4952-8bf7-8745d5605587 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eced5f58c74d41aabe3e7e48cfbc5f52'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:38.648454 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d85c1545-08d9-4952-8bf7-8745d5605587 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eced5f58c74d41aabe3e7e48cfbc5f52'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:38.656374 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d85c1545-08d9-4952-8bf7-8745d5605587 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eced5f58c74d41aabe3e7e48cfbc5f52'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:38.656980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d85c1545-08d9-4952-8bf7-8745d5605587 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eced5f58c74d41aabe3e7e48cfbc5f52'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:38.670160 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d85c1545-08d9-4952-8bf7-8745d5605587 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eced5f58c74d41aabe3e7e48cfbc5f52'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:38.676823 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1052/2108] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:39:38 2022] DELETE /identity/v3/users/eced5f58c74d41aabe3e7e48cfbc5f52 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:38.944791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bf3e0-21ad-460c-aa95-93b7debefa34 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:38.947652 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d27bf3e0-21ad-460c-aa95-93b7debefa34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:38.962964 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bf3e0-21ad-460c-aa95-93b7debefa34 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:39.024927 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a6298-5b08-4ae0-ae43-60de7406e9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:39.025938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a6298-5b08-4ae0-ae43-60de7406e9d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:39.026986 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a6298-5b08-4ae0-ae43-60de7406e9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:39.027414 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a6298-5b08-4ae0-ae43-60de7406e9d5 admin admin] PATH_INFO: `/v3/projects/eca6d0c2ad714890909e1fd7426aac6e` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:39.028697 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a6298-5b08-4ae0-ae43-60de7406e9d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eca6d0c2ad714890909e1fd7426aac6e)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:39.030590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a6298-5b08-4ae0-ae43-60de7406e9d5 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:39.081839 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-300a6298-5b08-4ae0-ae43-60de7406e9d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eca6d0c2ad714890909e1fd7426aac6e', 'request_id': 'req-300a6298-5b08-4ae0-ae43-60de7406e9d5'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:39.090282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-300a6298-5b08-4ae0-ae43-60de7406e9d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eca6d0c2ad714890909e1fd7426aac6e', 'request_id': 'req-300a6298-5b08-4ae0-ae43-60de7406e9d5'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:39.096679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-300a6298-5b08-4ae0-ae43-60de7406e9d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eca6d0c2ad714890909e1fd7426aac6e', 'request_id': 'req-300a6298-5b08-4ae0-ae43-60de7406e9d5'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:39.108811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-300a6298-5b08-4ae0-ae43-60de7406e9d5 admin admin] The token cache is being invalidate because project eca6d0c2ad714890909e1fd7426aac6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:39:39.109103 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-300a6298-5b08-4ae0-ae43-60de7406e9d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:39.109762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-300a6298-5b08-4ae0-ae43-60de7406e9d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:39.111300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1057/2109] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:39:38 2022] DELETE /identity/v3/projects/eca6d0c2ad714890909e1fd7426aac6e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:39.119442 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7fc16-5736-43bc-9520-edc3e7fd5cbc None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:39.119992 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7fc16-5736-43bc-9520-edc3e7fd5cbc None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:39.120086 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7fc16-5736-43bc-9520-edc3e7fd5cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:39.137937 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-fdf7fc16-5736-43bc-9520-edc3e7fd5cbc None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:39.139275 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-fdf7fc16-5736-43bc-9520-edc3e7fd5cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:39.209184 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1053/2110] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:39.217541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a82b8d-e1c2-4277-92cc-7ec2b24491c2 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:39.217832 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a82b8d-e1c2-4277-92cc-7ec2b24491c2 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:39.218106 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a82b8d-e1c2-4277-92cc-7ec2b24491c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:39.229582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7fc16-5736-43bc-9520-edc3e7fd5cbc None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:39.231953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-fdf7fc16-5736-43bc-9520-edc3e7fd5cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:39.237283 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-69a82b8d-e1c2-4277-92cc-7ec2b24491c2 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:39.238463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-69a82b8d-e1c2-4277-92cc-7ec2b24491c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:39.247836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1058/2111] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:39.252434 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a82b8d-e1c2-4277-92cc-7ec2b24491c2 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:39.264665 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a82b8d-e1c2-4277-92cc-7ec2b24491c2 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:39.271302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df2e08-fd42-4ddd-aabd-4f7693f34a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:39.273079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df2e08-fd42-4ddd-aabd-4f7693f34a0c admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:39.273427 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df2e08-fd42-4ddd-aabd-4f7693f34a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:39.273779 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df2e08-fd42-4ddd-aabd-4f7693f34a0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:39.274580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df2e08-fd42-4ddd-aabd-4f7693f34a0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:39:39.274957 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df2e08-fd42-4ddd-aabd-4f7693f34a0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:39.277300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df2e08-fd42-4ddd-aabd-4f7693f34a0c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:39.295972 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1059/2112] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:39:39.297690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df2e08-fd42-4ddd-aabd-4f7693f34a0c admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:39.304586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:39.305441 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7fc16-5736-43bc-9520-edc3e7fd5cbc None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:39.309107 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df2e08-fd42-4ddd-aabd-4f7693f34a0c admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:39.310865 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b826b-f297-4fd6-a9ce-c2a5e9db4f1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:39.311555 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b826b-f297-4fd6-a9ce-c2a5e9db4f1c service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:39.311917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b826b-f297-4fd6-a9ce-c2a5e9db4f1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:39.312119 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b826b-f297-4fd6-a9ce-c2a5e9db4f1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:39.314397 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-079b826b-f297-4fd6-a9ce-c2a5e9db4f1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:39.318128 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac8a45-be0a-4b2e-8d95-01d8aeda4773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:39.319291 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac8a45-be0a-4b2e-8d95-01d8aeda4773 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:39.319401 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac8a45-be0a-4b2e-8d95-01d8aeda4773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:39.319734 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac8a45-be0a-4b2e-8d95-01d8aeda4773 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:39.321151 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac8a45-be0a-4b2e-8d95-01d8aeda4773 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:39.324550 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac8a45-be0a-4b2e-8d95-01d8aeda4773 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:39.357221 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1060/2113] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:39:39 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:39.361412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac8a45-be0a-4b2e-8d95-01d8aeda4773 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:39.380579 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:39.400801 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b826b-f297-4fd6-a9ce-c2a5e9db4f1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:39.403419 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b826b-f297-4fd6-a9ce-c2a5e9db4f1c service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:39.434226 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1054/2114] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:39:39.450325 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac8a45-be0a-4b2e-8d95-01d8aeda4773 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:39.455679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74330e-439a-425c-a91f-78d535669e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:39.456808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74330e-439a-425c-a91f-78d535669e68 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:39.457080 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74330e-439a-425c-a91f-78d535669e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:39.457352 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74330e-439a-425c-a91f-78d535669e68 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:39.458142 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74330e-439a-425c-a91f-78d535669e68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:39.460077 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74330e-439a-425c-a91f-78d535669e68 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:39.489561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1061/2115] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:39:39 2022] POST /identity/v3/users => generated 294 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:39:39.493687 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b826b-f297-4fd6-a9ce-c2a5e9db4f1c service cinder] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:39.523331 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b826b-f297-4fd6-a9ce-c2a5e9db4f1c service cinder] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:39.535947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e0b6b-80cd-4008-b975-78cf0b6c40f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:39.536725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e0b6b-80cd-4008-b975-78cf0b6c40f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:39.537063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e0b6b-80cd-4008-b975-78cf0b6c40f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:39.537351 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e0b6b-80cd-4008-b975-78cf0b6c40f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:39.538178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e0b6b-80cd-4008-b975-78cf0b6c40f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:39.539890 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e0b6b-80cd-4008-b975-78cf0b6c40f0 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:39.555901 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1055/2116] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:39:39.557030 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74330e-439a-425c-a91f-78d535669e68 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:39.568808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74330e-439a-425c-a91f-78d535669e68 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:39.575732 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84192e8-5e44-4ce1-95d4-ccbf0268a60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:39.576929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84192e8-5e44-4ce1-95d4-ccbf0268a60f admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:39.577227 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84192e8-5e44-4ce1-95d4-ccbf0268a60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:39.577494 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84192e8-5e44-4ce1-95d4-ccbf0268a60f admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:39.578227 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84192e8-5e44-4ce1-95d4-ccbf0268a60f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:39.580015 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84192e8-5e44-4ce1-95d4-ccbf0268a60f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:39.591936 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1062/2117] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:39:39.597091 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e0b6b-80cd-4008-b975-78cf0b6c40f0 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:39.608792 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e0b6b-80cd-4008-b975-78cf0b6c40f0 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:39.614586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a7a9a-0a22-4e31-8783-867b2d2495b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:39.615604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a7a9a-0a22-4e31-8783-867b2d2495b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:39.615808 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a7a9a-0a22-4e31-8783-867b2d2495b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:39.616052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a7a9a-0a22-4e31-8783-867b2d2495b3 admin admin] PATH_INFO: `/v3/projects/2cb30f0903b546d6b267c9b7423c3b60/users/eadeaa464d504319939d5fa3bf9501e3/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:39.633051 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a7a9a-0a22-4e31-8783-867b2d2495b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cb30f0903b546d6b267c9b7423c3b60, user_id=eadeaa464d504319939d5fa3bf9501e3, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:39.636964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a7a9a-0a22-4e31-8783-867b2d2495b3 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:39.671272 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1056/2118] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:39 2022] PUT /identity/v3/projects/2cb30f0903b546d6b267c9b7423c3b60/users/eadeaa464d504319939d5fa3bf9501e3/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:39.681894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820d3a9-1585-4fdf-a0d3-8b7a3206adf5 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:39.684305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820d3a9-1585-4fdf-a0d3-8b7a3206adf5 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:39.684599 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820d3a9-1585-4fdf-a0d3-8b7a3206adf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:39.716314 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-d820d3a9-1585-4fdf-a0d3-8b7a3206adf5 None None] MFA Rules not processed for user `eadeaa464d504319939d5fa3bf9501e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:39.718047 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d820d3a9-1585-4fdf-a0d3-8b7a3206adf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:39.804870 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1063/2119] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:39.810206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a7a9a-0a22-4e31-8783-867b2d2495b3 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:39.882117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a7a9a-0a22-4e31-8783-867b2d2495b3 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:39.886884 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1563f9-fa2e-4431-80cd-bbaedf8ec2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:39.887541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1563f9-fa2e-4431-80cd-bbaedf8ec2c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:39.887858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1563f9-fa2e-4431-80cd-bbaedf8ec2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:39.888138 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1563f9-fa2e-4431-80cd-bbaedf8ec2c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:39.888845 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1563f9-fa2e-4431-80cd-bbaedf8ec2c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:39.890923 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1563f9-fa2e-4431-80cd-bbaedf8ec2c0 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:39.917273 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1057/2120] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:39:39 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:39.921619 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820d3a9-1585-4fdf-a0d3-8b7a3206adf5 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:39.932231 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820d3a9-1585-4fdf-a0d3-8b7a3206adf5 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:39.979850 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:39.991109 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdf519-3105-4879-bd55-e0ba188fcfbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:39.991793 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdf519-3105-4879-bd55-e0ba188fcfbb admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:39.992102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdf519-3105-4879-bd55-e0ba188fcfbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:39.992379 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdf519-3105-4879-bd55-e0ba188fcfbb admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:39.993067 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fdf519-3105-4879-bd55-e0ba188fcfbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:39.995659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fdf519-3105-4879-bd55-e0ba188fcfbb admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:40.014676 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1064/2121] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:39:39 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:40.018713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1563f9-fa2e-4431-80cd-bbaedf8ec2c0 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:40.030301 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1563f9-fa2e-4431-80cd-bbaedf8ec2c0 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:40.034920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066d2ef-b4b9-42bb-b212-ec281647f42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:40.035601 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066d2ef-b4b9-42bb-b212-ec281647f42b admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:40.035914 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066d2ef-b4b9-42bb-b212-ec281647f42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:40.036189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066d2ef-b4b9-42bb-b212-ec281647f42b admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:40.036908 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066d2ef-b4b9-42bb-b212-ec281647f42b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:40.038499 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066d2ef-b4b9-42bb-b212-ec281647f42b admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:40.049537 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1058/2122] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:40.053527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdf519-3105-4879-bd55-e0ba188fcfbb admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:40.063352 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdf519-3105-4879-bd55-e0ba188fcfbb admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:40.068051 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a568f6d-1b93-4cec-a91a-831d6c6423da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:40.068925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a568f6d-1b93-4cec-a91a-831d6c6423da admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:40.069206 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a568f6d-1b93-4cec-a91a-831d6c6423da admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:40.069479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a568f6d-1b93-4cec-a91a-831d6c6423da admin admin] PATH_INFO: `/v3/projects/ea97ca72e99543fdae451a04408af7ca/users/209435f5d326475081332378dc6b36f8/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:40.089147 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a568f6d-1b93-4cec-a91a-831d6c6423da admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea97ca72e99543fdae451a04408af7ca, user_id=209435f5d326475081332378dc6b36f8, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:40.092356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a568f6d-1b93-4cec-a91a-831d6c6423da admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:40.107996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1065/2123] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:40 2022] PUT /identity/v3/projects/ea97ca72e99543fdae451a04408af7ca/users/209435f5d326475081332378dc6b36f8/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:40.111703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066d2ef-b4b9-42bb-b212-ec281647f42b admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:40.122477 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066d2ef-b4b9-42bb-b212-ec281647f42b admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:40.181394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db650ea6-5f67-47b7-80df-1d331be49066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:40.182134 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db650ea6-5f67-47b7-80df-1d331be49066 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:40.182432 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db650ea6-5f67-47b7-80df-1d331be49066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:40.182698 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db650ea6-5f67-47b7-80df-1d331be49066 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:40.183413 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db650ea6-5f67-47b7-80df-1d331be49066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:40.185104 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db650ea6-5f67-47b7-80df-1d331be49066 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:40.194548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1059/2124] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:40.198888 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a568f6d-1b93-4cec-a91a-831d6c6423da admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:40.208990 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a568f6d-1b93-4cec-a91a-831d6c6423da admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:40.214098 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da8452-9039-4fd9-bc41-3ff09d6fe70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:40.214977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da8452-9039-4fd9-bc41-3ff09d6fe70c admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:40.215255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da8452-9039-4fd9-bc41-3ff09d6fe70c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:40.215527 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da8452-9039-4fd9-bc41-3ff09d6fe70c admin admin] PATH_INFO: `/v3/projects/ea97ca72e99543fdae451a04408af7ca/users/209435f5d326475081332378dc6b36f8/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:40.217555 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da8452-9039-4fd9-bc41-3ff09d6fe70c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea97ca72e99543fdae451a04408af7ca, user_id=209435f5d326475081332378dc6b36f8, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:40.220223 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da8452-9039-4fd9-bc41-3ff09d6fe70c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:40.231370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1066/2125] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:40 2022] PUT /identity/v3/projects/ea97ca72e99543fdae451a04408af7ca/users/209435f5d326475081332378dc6b36f8/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:40.241368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f3023-c2c2-426b-bf3a-819647aff454 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:40.241656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f3023-c2c2-426b-bf3a-819647aff454 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:40.241930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f3023-c2c2-426b-bf3a-819647aff454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:40.270247 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-ae7f3023-c2c2-426b-bf3a-819647aff454 None None] MFA Rules not processed for user `209435f5d326475081332378dc6b36f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:40.271523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ae7f3023-c2c2-426b-bf3a-819647aff454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:40.285142 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:40.347243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1060/2126] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:39:40.358869 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da8452-9039-4fd9-bc41-3ff09d6fe70c admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:40.434239 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da8452-9039-4fd9-bc41-3ff09d6fe70c admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:40.437948 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde6664-9283-4e29-9416-c8a83ae8d133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:40.438594 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde6664-9283-4e29-9416-c8a83ae8d133 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:40.438876 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde6664-9283-4e29-9416-c8a83ae8d133 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:40.439148 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde6664-9283-4e29-9416-c8a83ae8d133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:40.448533 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde6664-9283-4e29-9416-c8a83ae8d133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:40.449953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde6664-9283-4e29-9416-c8a83ae8d133 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:40.459738 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1067/2127] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:39:40.528717 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f3023-c2c2-426b-bf3a-819647aff454 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:40.617161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f3023-c2c2-426b-bf3a-819647aff454 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:40.621950 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d649cb-e978-4f5d-a2b4-ce15ec573def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:40.622818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d649cb-e978-4f5d-a2b4-ce15ec573def admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:40.623102 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d649cb-e978-4f5d-a2b4-ce15ec573def admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:40.623380 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d649cb-e978-4f5d-a2b4-ce15ec573def admin admin] PATH_INFO: `/v3/users/eadeaa464d504319939d5fa3bf9501e3` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:40.624612 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d649cb-e978-4f5d-a2b4-ce15ec573def admin admin] RBAC: Authorizing `identity:delete_user(user_id=eadeaa464d504319939d5fa3bf9501e3)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:40.626520 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d649cb-e978-4f5d-a2b4-ce15ec573def admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:40.673421 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b2d649cb-e978-4f5d-a2b4-ce15ec573def admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eadeaa464d504319939d5fa3bf9501e3'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:40.680187 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b2d649cb-e978-4f5d-a2b4-ce15ec573def admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eadeaa464d504319939d5fa3bf9501e3'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:40.687283 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b2d649cb-e978-4f5d-a2b4-ce15ec573def admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eadeaa464d504319939d5fa3bf9501e3'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:40.687921 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b2d649cb-e978-4f5d-a2b4-ce15ec573def admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eadeaa464d504319939d5fa3bf9501e3'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:40.700290 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b2d649cb-e978-4f5d-a2b4-ce15ec573def admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eadeaa464d504319939d5fa3bf9501e3'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:40.706130 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1061/2128] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:39:40 2022] DELETE /identity/v3/users/eadeaa464d504319939d5fa3bf9501e3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:40.710726 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650526da-a1cb-4323-9a63-f19416e447a1 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:40.711019 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650526da-a1cb-4323-9a63-f19416e447a1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:40.711297 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650526da-a1cb-4323-9a63-f19416e447a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:40.729081 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-650526da-a1cb-4323-9a63-f19416e447a1 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:40.730382 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-650526da-a1cb-4323-9a63-f19416e447a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:40.788958 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1068/2129] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:40.798600 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d649cb-e978-4f5d-a2b4-ce15ec573def admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:40.800020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-b2d649cb-e978-4f5d-a2b4-ce15ec573def admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:40.832424 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:40.862446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d649cb-e978-4f5d-a2b4-ce15ec573def admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:40.866444 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452484d5-8eda-4398-9db4-35d6ae83687f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:40.867094 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452484d5-8eda-4398-9db4-35d6ae83687f service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:40.867379 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452484d5-8eda-4398-9db4-35d6ae83687f service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:40.867655 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452484d5-8eda-4398-9db4-35d6ae83687f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:40.878460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452484d5-8eda-4398-9db4-35d6ae83687f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:40.879901 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452484d5-8eda-4398-9db4-35d6ae83687f service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:40.890601 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1062/2130] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:39:41.259455 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650526da-a1cb-4323-9a63-f19416e447a1 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:41.260972 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-650526da-a1cb-4323-9a63-f19416e447a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:41.275331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650526da-a1cb-4323-9a63-f19416e447a1 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:41.281044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2917c1-a0ad-4917-abec-1dc4e1c0e0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:41.282119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2917c1-a0ad-4917-abec-1dc4e1c0e0a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:41.282636 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2917c1-a0ad-4917-abec-1dc4e1c0e0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:41.282928 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2917c1-a0ad-4917-abec-1dc4e1c0e0a8 admin admin] PATH_INFO: `/v3/projects/2cb30f0903b546d6b267c9b7423c3b60` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:41.284360 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2917c1-a0ad-4917-abec-1dc4e1c0e0a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cb30f0903b546d6b267c9b7423c3b60)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:41.286546 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2917c1-a0ad-4917-abec-1dc4e1c0e0a8 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:41.330210 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:41.338286 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-6c2917c1-a0ad-4917-abec-1dc4e1c0e0a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cb30f0903b546d6b267c9b7423c3b60', 'request_id': 'req-6c2917c1-a0ad-4917-abec-1dc4e1c0e0a8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:41.347194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-6c2917c1-a0ad-4917-abec-1dc4e1c0e0a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cb30f0903b546d6b267c9b7423c3b60', 'request_id': 'req-6c2917c1-a0ad-4917-abec-1dc4e1c0e0a8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:41.356450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-6c2917c1-a0ad-4917-abec-1dc4e1c0e0a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cb30f0903b546d6b267c9b7423c3b60', 'request_id': 'req-6c2917c1-a0ad-4917-abec-1dc4e1c0e0a8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:41.367828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-6c2917c1-a0ad-4917-abec-1dc4e1c0e0a8 admin admin] The token cache is being invalidate because project 2cb30f0903b546d6b267c9b7423c3b60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:39:41.368112 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-6c2917c1-a0ad-4917-abec-1dc4e1c0e0a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:41.368761 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-6c2917c1-a0ad-4917-abec-1dc4e1c0e0a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:41.370893 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1069/2131] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:39:41 2022] DELETE /identity/v3/projects/2cb30f0903b546d6b267c9b7423c3b60 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:41.374917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452484d5-8eda-4398-9db4-35d6ae83687f service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:41.384873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452484d5-8eda-4398-9db4-35d6ae83687f service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:41.443060 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4576af-46e6-40e0-acef-d86ffab28cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:41.443974 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4576af-46e6-40e0-acef-d86ffab28cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:41.444262 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4576af-46e6-40e0-acef-d86ffab28cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:41.444540 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4576af-46e6-40e0-acef-d86ffab28cb5 admin admin] PATH_INFO: `/v3/users/209435f5d326475081332378dc6b36f8` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:41.445734 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4576af-46e6-40e0-acef-d86ffab28cb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=209435f5d326475081332378dc6b36f8)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:41.447640 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4576af-46e6-40e0-acef-d86ffab28cb5 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:41.493295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1d4576af-46e6-40e0-acef-d86ffab28cb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '209435f5d326475081332378dc6b36f8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:41.501797 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1d4576af-46e6-40e0-acef-d86ffab28cb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '209435f5d326475081332378dc6b36f8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:41.508639 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1d4576af-46e6-40e0-acef-d86ffab28cb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '209435f5d326475081332378dc6b36f8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:41.509273 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1d4576af-46e6-40e0-acef-d86ffab28cb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '209435f5d326475081332378dc6b36f8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:41.521877 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1d4576af-46e6-40e0-acef-d86ffab28cb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '209435f5d326475081332378dc6b36f8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:41.528439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1063/2132] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:39:41 2022] DELETE /identity/v3/users/209435f5d326475081332378dc6b36f8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:41.815476 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2917c1-a0ad-4917-abec-1dc4e1c0e0a8 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:41.817083 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6c2917c1-a0ad-4917-abec-1dc4e1c0e0a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:41.838375 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2917c1-a0ad-4917-abec-1dc4e1c0e0a8 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:41.900232 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed121f7c-e806-4edb-9c68-5d89c301b29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:41.901101 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed121f7c-e806-4edb-9c68-5d89c301b29f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:41.901379 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed121f7c-e806-4edb-9c68-5d89c301b29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:41.901683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed121f7c-e806-4edb-9c68-5d89c301b29f admin admin] PATH_INFO: `/v3/projects/ea97ca72e99543fdae451a04408af7ca` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:41.902866 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed121f7c-e806-4edb-9c68-5d89c301b29f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea97ca72e99543fdae451a04408af7ca)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:41.904812 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed121f7c-e806-4edb-9c68-5d89c301b29f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:41.950469 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ed121f7c-e806-4edb-9c68-5d89c301b29f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea97ca72e99543fdae451a04408af7ca', 'request_id': 'req-ed121f7c-e806-4edb-9c68-5d89c301b29f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:41.957859 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ed121f7c-e806-4edb-9c68-5d89c301b29f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea97ca72e99543fdae451a04408af7ca', 'request_id': 'req-ed121f7c-e806-4edb-9c68-5d89c301b29f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:41.964227 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ed121f7c-e806-4edb-9c68-5d89c301b29f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea97ca72e99543fdae451a04408af7ca', 'request_id': 'req-ed121f7c-e806-4edb-9c68-5d89c301b29f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:41.979452 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ed121f7c-e806-4edb-9c68-5d89c301b29f admin admin] The token cache is being invalidate because project ea97ca72e99543fdae451a04408af7ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:39:41.979916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ed121f7c-e806-4edb-9c68-5d89c301b29f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:41.980999 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ed121f7c-e806-4edb-9c68-5d89c301b29f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:41.983229 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1070/2133] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:39:41 2022] DELETE /identity/v3/projects/ea97ca72e99543fdae451a04408af7ca => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:41.988305 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31894b6-95cf-437f-a0d2-c0998e544626 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:41.988597 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31894b6-95cf-437f-a0d2-c0998e544626 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:41.988878 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31894b6-95cf-437f-a0d2-c0998e544626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:42.009265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-b31894b6-95cf-437f-a0d2-c0998e544626 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:42.010750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-b31894b6-95cf-437f-a0d2-c0998e544626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:42.055369 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:42.072855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1064/2134] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:42.082186 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff682804-52c1-40ad-b931-8abfbe9ad39d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:42.082474 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff682804-52c1-40ad-b931-8abfbe9ad39d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:42.082747 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff682804-52c1-40ad-b931-8abfbe9ad39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:42.100960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ff682804-52c1-40ad-b931-8abfbe9ad39d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:42.102144 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ff682804-52c1-40ad-b931-8abfbe9ad39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:42.110850 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1071/2135] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:42.115149 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31894b6-95cf-437f-a0d2-c0998e544626 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:42.125672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31894b6-95cf-437f-a0d2-c0998e544626 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:42.130459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bea0b-96c7-41d9-8d61-d157a33b937a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:42.131137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bea0b-96c7-41d9-8d61-d157a33b937a admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:42.131483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bea0b-96c7-41d9-8d61-d157a33b937a admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:42.131856 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bea0b-96c7-41d9-8d61-d157a33b937a admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:42.132652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bea0b-96c7-41d9-8d61-d157a33b937a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:39:42.132968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bea0b-96c7-41d9-8d61-d157a33b937a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:42.134559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bea0b-96c7-41d9-8d61-d157a33b937a admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:42.146689 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1065/2136] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:39:42.150905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff682804-52c1-40ad-b931-8abfbe9ad39d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:42.160345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff682804-52c1-40ad-b931-8abfbe9ad39d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:42.165017 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4a4bb-855c-48e9-9332-48470ce338b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:42.165608 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4a4bb-855c-48e9-9332-48470ce338b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:42.165885 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4a4bb-855c-48e9-9332-48470ce338b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:42.166155 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4a4bb-855c-48e9-9332-48470ce338b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:42.167563 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4a4bb-855c-48e9-9332-48470ce338b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:42.169609 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4a4bb-855c-48e9-9332-48470ce338b1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:42.195168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1072/2137] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:39:42 2022] POST /identity/v3/projects => generated 409 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:42.200205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bea0b-96c7-41d9-8d61-d157a33b937a admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:42.210077 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bea0b-96c7-41d9-8d61-d157a33b937a admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:42.268663 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e184f58-6b62-42de-be0a-a8970f433af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:42.269327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e184f58-6b62-42de-be0a-a8970f433af6 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:42.269866 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e184f58-6b62-42de-be0a-a8970f433af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:42.270143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e184f58-6b62-42de-be0a-a8970f433af6 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:42.270824 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e184f58-6b62-42de-be0a-a8970f433af6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:42.272778 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e184f58-6b62-42de-be0a-a8970f433af6 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:42.293273 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1066/2138] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:39:42 2022] POST /identity/v3/users => generated 305 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:39:42.297340 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4a4bb-855c-48e9-9332-48470ce338b1 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:42.308047 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4a4bb-855c-48e9-9332-48470ce338b1 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:42.312658 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3d622-028f-4cbb-9c19-d25b768b1870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:42.313309 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3d622-028f-4cbb-9c19-d25b768b1870 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:42.313591 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3d622-028f-4cbb-9c19-d25b768b1870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:42.313865 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3d622-028f-4cbb-9c19-d25b768b1870 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:42.314588 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3d622-028f-4cbb-9c19-d25b768b1870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:42.316221 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3d622-028f-4cbb-9c19-d25b768b1870 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:42.326948 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1073/2139] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:42.330846 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e184f58-6b62-42de-be0a-a8970f433af6 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:42.340966 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e184f58-6b62-42de-be0a-a8970f433af6 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:42.345634 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451de36c-3b02-407d-aa6c-986fd8b4ad2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:42.346285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451de36c-3b02-407d-aa6c-986fd8b4ad2b admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:42.346573 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451de36c-3b02-407d-aa6c-986fd8b4ad2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:42.347529 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451de36c-3b02-407d-aa6c-986fd8b4ad2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:42.348290 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451de36c-3b02-407d-aa6c-986fd8b4ad2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:42.349876 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451de36c-3b02-407d-aa6c-986fd8b4ad2b admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:42.360040 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1067/2140] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:42.364115 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3d622-028f-4cbb-9c19-d25b768b1870 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:42.374647 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3d622-028f-4cbb-9c19-d25b768b1870 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:42.379389 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04dabb3-1e89-4862-9e28-ddbff22f0005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:42.380308 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04dabb3-1e89-4862-9e28-ddbff22f0005 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:42.380645 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04dabb3-1e89-4862-9e28-ddbff22f0005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:42.380863 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04dabb3-1e89-4862-9e28-ddbff22f0005 admin admin] PATH_INFO: `/v3/projects/5a29ea01c9a24cd182875abfde0eef67/users/0fad66f1434a44bb9a22a78bf3a8a3ab/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:42.397191 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04dabb3-1e89-4862-9e28-ddbff22f0005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a29ea01c9a24cd182875abfde0eef67, user_id=0fad66f1434a44bb9a22a78bf3a8a3ab, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:42.399906 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04dabb3-1e89-4862-9e28-ddbff22f0005 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:42.403139 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:42.416645 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1074/2141] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:42 2022] PUT /identity/v3/projects/5a29ea01c9a24cd182875abfde0eef67/users/0fad66f1434a44bb9a22a78bf3a8a3ab/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:42.426969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afde50-e6e6-4daa-bda6-b3eff758b831 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:42.427417 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afde50-e6e6-4daa-bda6-b3eff758b831 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:42.427506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afde50-e6e6-4daa-bda6-b3eff758b831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:42.454346 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-c0afde50-e6e6-4daa-bda6-b3eff758b831 None None] MFA Rules not processed for user `0fad66f1434a44bb9a22a78bf3a8a3ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:42.455525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-c0afde50-e6e6-4daa-bda6-b3eff758b831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:42.521129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1068/2142] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:42 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:42.525423 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04dabb3-1e89-4862-9e28-ddbff22f0005 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:42.585690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04dabb3-1e89-4862-9e28-ddbff22f0005 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:42.590371 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f54683-0476-45ca-aa28-2165361f1dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:42.591012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f54683-0476-45ca-aa28-2165361f1dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:42.591290 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f54683-0476-45ca-aa28-2165361f1dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:42.591561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f54683-0476-45ca-aa28-2165361f1dd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:42.592280 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f54683-0476-45ca-aa28-2165361f1dd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:42.594158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f54683-0476-45ca-aa28-2165361f1dd7 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:42.620675 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1075/2143] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:39:42 2022] POST /identity/v3/projects => generated 409 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:42.624756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afde50-e6e6-4daa-bda6-b3eff758b831 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:42.636116 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afde50-e6e6-4daa-bda6-b3eff758b831 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:42.680992 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:42.695893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c6a43-a28b-44a0-973d-9aaa4680915d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:42.696559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c6a43-a28b-44a0-973d-9aaa4680915d admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:42.696840 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c6a43-a28b-44a0-973d-9aaa4680915d admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:42.697117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c6a43-a28b-44a0-973d-9aaa4680915d admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:42.698322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c6a43-a28b-44a0-973d-9aaa4680915d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:42.700819 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c6a43-a28b-44a0-973d-9aaa4680915d admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:42.722499 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1069/2144] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:39:42 2022] POST /identity/v3/users => generated 311 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:42.726504 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f54683-0476-45ca-aa28-2165361f1dd7 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:42.737122 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f54683-0476-45ca-aa28-2165361f1dd7 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:42.741682 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8abaf-9ff0-49bd-bd8c-1aca2ba64c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:42.742329 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8abaf-9ff0-49bd-bd8c-1aca2ba64c1e admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:42.742608 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8abaf-9ff0-49bd-bd8c-1aca2ba64c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:42.742879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8abaf-9ff0-49bd-bd8c-1aca2ba64c1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:42.743595 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8abaf-9ff0-49bd-bd8c-1aca2ba64c1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:42.745720 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8abaf-9ff0-49bd-bd8c-1aca2ba64c1e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:42.755925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1076/2145] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:39:42.760021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c6a43-a28b-44a0-973d-9aaa4680915d admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:42.771036 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c6a43-a28b-44a0-973d-9aaa4680915d admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:42.775671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e5954-6cd2-43a6-98c6-bd489827646c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:42.777193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e5954-6cd2-43a6-98c6-bd489827646c admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:42.777507 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e5954-6cd2-43a6-98c6-bd489827646c admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:42.777784 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e5954-6cd2-43a6-98c6-bd489827646c admin admin] PATH_INFO: `/v3/projects/cce458aea2784ea895855be4c1fce6d1/users/af9ac825c8b54daa8e6a5aa9c0592471/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:42.792493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e5954-6cd2-43a6-98c6-bd489827646c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cce458aea2784ea895855be4c1fce6d1, user_id=af9ac825c8b54daa8e6a5aa9c0592471, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:42.795122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e5954-6cd2-43a6-98c6-bd489827646c admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:42.808265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1070/2146] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:42 2022] PUT /identity/v3/projects/cce458aea2784ea895855be4c1fce6d1/users/af9ac825c8b54daa8e6a5aa9c0592471/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:42.812275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8abaf-9ff0-49bd-bd8c-1aca2ba64c1e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:42.822310 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8abaf-9ff0-49bd-bd8c-1aca2ba64c1e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:42.884098 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2443f8c-3aed-4912-8e59-4463612e7fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:42.885919 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2443f8c-3aed-4912-8e59-4463612e7fbb admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:42.886475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2443f8c-3aed-4912-8e59-4463612e7fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:42.886837 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2443f8c-3aed-4912-8e59-4463612e7fbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:42.887568 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2443f8c-3aed-4912-8e59-4463612e7fbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:42.889206 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2443f8c-3aed-4912-8e59-4463612e7fbb admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:42.899870 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1077/2147] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:42.905140 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e5954-6cd2-43a6-98c6-bd489827646c admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:42.916277 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e5954-6cd2-43a6-98c6-bd489827646c admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:42.921695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e60eb4-257e-46a0-aa21-887beed6d41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:42.922582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e60eb4-257e-46a0-aa21-887beed6d41e admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:42.922863 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e60eb4-257e-46a0-aa21-887beed6d41e admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:42.923134 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e60eb4-257e-46a0-aa21-887beed6d41e admin admin] PATH_INFO: `/v3/projects/cce458aea2784ea895855be4c1fce6d1/users/af9ac825c8b54daa8e6a5aa9c0592471/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:42.925180 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e60eb4-257e-46a0-aa21-887beed6d41e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cce458aea2784ea895855be4c1fce6d1, user_id=af9ac825c8b54daa8e6a5aa9c0592471, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:42.927800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e60eb4-257e-46a0-aa21-887beed6d41e admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:42.938605 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1071/2148] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:42 2022] PUT /identity/v3/projects/cce458aea2784ea895855be4c1fce6d1/users/af9ac825c8b54daa8e6a5aa9c0592471/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:42.949241 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5517527-66c3-41da-834a-4338d8826871 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:42.949525 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5517527-66c3-41da-834a-4338d8826871 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:42.949803 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5517527-66c3-41da-834a-4338d8826871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:42.977049 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-d5517527-66c3-41da-834a-4338d8826871 None None] MFA Rules not processed for user `af9ac825c8b54daa8e6a5aa9c0592471`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:42.978274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d5517527-66c3-41da-834a-4338d8826871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:43.030884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:43.054881 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1078/2149] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:42 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:43.067620 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e60eb4-257e-46a0-aa21-887beed6d41e admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:43.069396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-63e60eb4-257e-46a0-aa21-887beed6d41e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:43.129398 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e60eb4-257e-46a0-aa21-887beed6d41e admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:43.133507 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b857ff5-17f7-4b32-8735-d6b118e78a41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:43.134142 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b857ff5-17f7-4b32-8735-d6b118e78a41 service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:43.134441 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b857ff5-17f7-4b32-8735-d6b118e78a41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:43.134713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b857ff5-17f7-4b32-8735-d6b118e78a41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:43.144347 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b857ff5-17f7-4b32-8735-d6b118e78a41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:43.145973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b857ff5-17f7-4b32-8735-d6b118e78a41 service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:43.156641 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1072/2150] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:43 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:39:43.307541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5517527-66c3-41da-834a-4338d8826871 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:43.317521 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5517527-66c3-41da-834a-4338d8826871 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:43.321470 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69247a4e-6a26-4a2f-9a51-d0b6e552e318 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:43.322122 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69247a4e-6a26-4a2f-9a51-d0b6e552e318 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:43.322404 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69247a4e-6a26-4a2f-9a51-d0b6e552e318 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:43.323299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69247a4e-6a26-4a2f-9a51-d0b6e552e318 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:43.395923 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69247a4e-6a26-4a2f-9a51-d0b6e552e318 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:43.396913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69247a4e-6a26-4a2f-9a51-d0b6e552e318 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:43.410764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1079/2151] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:43 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:39:44.987440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b857ff5-17f7-4b32-8735-d6b118e78a41 service cinder] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:44.990478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-5b857ff5-17f7-4b32-8735-d6b118e78a41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:45.069850 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b857ff5-17f7-4b32-8735-d6b118e78a41 service cinder] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:45.076134 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a3866-fa6c-4c18-a736-f9f3ff47fd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:45.077159 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a3866-fa6c-4c18-a736-f9f3ff47fd89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:45.077599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a3866-fa6c-4c18-a736-f9f3ff47fd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:45.077930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a3866-fa6c-4c18-a736-f9f3ff47fd89 admin admin] PATH_INFO: `/v3/users/c57cf1e43a9f49edac6e006444eb73c9` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:45.079580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a3866-fa6c-4c18-a736-f9f3ff47fd89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c57cf1e43a9f49edac6e006444eb73c9)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:45.081845 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a3866-fa6c-4c18-a736-f9f3ff47fd89 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:45.133025 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:45.136985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-da0a3866-fa6c-4c18-a736-f9f3ff47fd89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c57cf1e43a9f49edac6e006444eb73c9'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:45.148163 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-da0a3866-fa6c-4c18-a736-f9f3ff47fd89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c57cf1e43a9f49edac6e006444eb73c9'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:45.158926 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-da0a3866-fa6c-4c18-a736-f9f3ff47fd89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c57cf1e43a9f49edac6e006444eb73c9'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:45.159620 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-da0a3866-fa6c-4c18-a736-f9f3ff47fd89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c57cf1e43a9f49edac6e006444eb73c9'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:45.174579 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-da0a3866-fa6c-4c18-a736-f9f3ff47fd89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c57cf1e43a9f49edac6e006444eb73c9'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:45.184442 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1073/2152] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:39:44 2022] DELETE /identity/v3/users/c57cf1e43a9f49edac6e006444eb73c9 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:45.190863 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb2d0a-857b-45f9-90d0-16cdb7108d19 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:45.191558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb2d0a-857b-45f9-90d0-16cdb7108d19 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:45.192243 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb2d0a-857b-45f9-90d0-16cdb7108d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:45.216309 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9fdb2d0a-857b-45f9-90d0-16cdb7108d19 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:45.217701 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9fdb2d0a-857b-45f9-90d0-16cdb7108d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:45.313461 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1080/2153] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:45.322449 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a3866-fa6c-4c18-a736-f9f3ff47fd89 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:45.324896 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-da0a3866-fa6c-4c18-a736-f9f3ff47fd89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:45.407395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a3866-fa6c-4c18-a736-f9f3ff47fd89 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:45.411015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3746a71-dede-4ac8-ac88-7e849ce06c16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:45.411655 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3746a71-dede-4ac8-ac88-7e849ce06c16 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:45.411965 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3746a71-dede-4ac8-ac88-7e849ce06c16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:45.412241 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3746a71-dede-4ac8-ac88-7e849ce06c16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:45.423633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3746a71-dede-4ac8-ac88-7e849ce06c16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:45.425311 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3746a71-dede-4ac8-ac88-7e849ce06c16 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:45.437740 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1074/2154] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:39:45.710154 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c8a6b-573e-46f5-b701-40a55269f40d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:45.710447 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c8a6b-573e-46f5-b701-40a55269f40d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:45.710727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c8a6b-573e-46f5-b701-40a55269f40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:45.731755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-6d1c8a6b-573e-46f5-b701-40a55269f40d None None] MFA Rules not processed for user `446b13b4661542c097afafb32443dae4`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:45.733822 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3746a71-dede-4ac8-ac88-7e849ce06c16 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:45.734715 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6d1c8a6b-573e-46f5-b701-40a55269f40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:45.737011 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-d3746a71-dede-4ac8-ac88-7e849ce06c16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:45.752121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3746a71-dede-4ac8-ac88-7e849ce06c16 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:45.756903 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba08ab-4d29-4f2b-81c7-4a45efaa7ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:45.757775 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba08ab-4d29-4f2b-81c7-4a45efaa7ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:45.758056 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba08ab-4d29-4f2b-81c7-4a45efaa7ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:45.758334 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba08ab-4d29-4f2b-81c7-4a45efaa7ae8 admin admin] PATH_INFO: `/v3/projects/b58baaa33f7940aab10e76c4423596d2` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:45.759571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba08ab-4d29-4f2b-81c7-4a45efaa7ae8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b58baaa33f7940aab10e76c4423596d2)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:45.762047 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba08ab-4d29-4f2b-81c7-4a45efaa7ae8 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:45.802470 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1081/2155] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:45.811459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e0ba08ab-4d29-4f2b-81c7-4a45efaa7ae8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b58baaa33f7940aab10e76c4423596d2', 'request_id': 'req-e0ba08ab-4d29-4f2b-81c7-4a45efaa7ae8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:45.817095 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e0ba08ab-4d29-4f2b-81c7-4a45efaa7ae8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b58baaa33f7940aab10e76c4423596d2', 'request_id': 'req-e0ba08ab-4d29-4f2b-81c7-4a45efaa7ae8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:45.825919 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e0ba08ab-4d29-4f2b-81c7-4a45efaa7ae8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b58baaa33f7940aab10e76c4423596d2', 'request_id': 'req-e0ba08ab-4d29-4f2b-81c7-4a45efaa7ae8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:45.837994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e0ba08ab-4d29-4f2b-81c7-4a45efaa7ae8 admin admin] The token cache is being invalidate because project b58baaa33f7940aab10e76c4423596d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:39:45.838863 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e0ba08ab-4d29-4f2b-81c7-4a45efaa7ae8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:45.839520 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e0ba08ab-4d29-4f2b-81c7-4a45efaa7ae8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:45.841242 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1075/2156] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:39:45 2022] DELETE /identity/v3/projects/b58baaa33f7940aab10e76c4423596d2 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:45.846145 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d242a5-30e8-4372-8dc2-7e4772a3d525 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:45.846433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d242a5-30e8-4372-8dc2-7e4772a3d525 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:45.846709 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d242a5-30e8-4372-8dc2-7e4772a3d525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:45.864807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b0d242a5-30e8-4372-8dc2-7e4772a3d525 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:45.866085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b0d242a5-30e8-4372-8dc2-7e4772a3d525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:45.897591 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:45.928607 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1082/2157] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:45.937763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b01ef5-164e-4823-a976-d36d40465381 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:45.938667 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b01ef5-164e-4823-a976-d36d40465381 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:45.938974 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b01ef5-164e-4823-a976-d36d40465381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:45.958050 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-70b01ef5-164e-4823-a976-d36d40465381 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:45.959278 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-70b01ef5-164e-4823-a976-d36d40465381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:45.968158 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1076/2158] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:45.972429 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d242a5-30e8-4372-8dc2-7e4772a3d525 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:45.985779 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d242a5-30e8-4372-8dc2-7e4772a3d525 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:45.990432 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688d966-9072-4458-a2bc-11829a7fb88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:45.991066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688d966-9072-4458-a2bc-11829a7fb88f admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:45.991346 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688d966-9072-4458-a2bc-11829a7fb88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:45.991618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688d966-9072-4458-a2bc-11829a7fb88f admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:45.992379 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688d966-9072-4458-a2bc-11829a7fb88f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:39:45.992681 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688d966-9072-4458-a2bc-11829a7fb88f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:45.994407 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688d966-9072-4458-a2bc-11829a7fb88f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:46.006936 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1083/2159] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:39:46.012358 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b01ef5-164e-4823-a976-d36d40465381 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:46.023583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b01ef5-164e-4823-a976-d36d40465381 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:46.029904 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ea710-c897-41b5-afb2-79f4d6431e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:46.030564 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ea710-c897-41b5-afb2-79f4d6431e67 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:46.030843 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ea710-c897-41b5-afb2-79f4d6431e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:46.031815 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ea710-c897-41b5-afb2-79f4d6431e67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:46.032536 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ea710-c897-41b5-afb2-79f4d6431e67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:46.034621 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ea710-c897-41b5-afb2-79f4d6431e67 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:46.066121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1077/2160] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:39:46 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:46.067575 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688d966-9072-4458-a2bc-11829a7fb88f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:46.078928 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688d966-9072-4458-a2bc-11829a7fb88f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:46.141077 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad60b3e-5f37-4752-846e-28a31a5f68b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:46.141847 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad60b3e-5f37-4752-846e-28a31a5f68b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:46.142227 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad60b3e-5f37-4752-846e-28a31a5f68b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:46.142572 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad60b3e-5f37-4752-846e-28a31a5f68b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:46.143345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad60b3e-5f37-4752-846e-28a31a5f68b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:46.145809 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad60b3e-5f37-4752-846e-28a31a5f68b5 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:46.170190 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1084/2161] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:39:46 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:46.171503 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ea710-c897-41b5-afb2-79f4d6431e67 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:46.190651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ea710-c897-41b5-afb2-79f4d6431e67 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:46.196436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283b94c-97fd-44fb-ac57-789807cf1a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:46.197326 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283b94c-97fd-44fb-ac57-789807cf1a08 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:46.198270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283b94c-97fd-44fb-ac57-789807cf1a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:46.198599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283b94c-97fd-44fb-ac57-789807cf1a08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:46.199844 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283b94c-97fd-44fb-ac57-789807cf1a08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:46.200927 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283b94c-97fd-44fb-ac57-789807cf1a08 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:46.212319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1078/2162] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:46.220103 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad60b3e-5f37-4752-846e-28a31a5f68b5 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:46.230564 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad60b3e-5f37-4752-846e-28a31a5f68b5 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:46.235646 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244c82e-2f55-4a9e-a735-7c019f6b0b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:46.236378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244c82e-2f55-4a9e-a735-7c019f6b0b20 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:46.236662 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244c82e-2f55-4a9e-a735-7c019f6b0b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:46.236938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244c82e-2f55-4a9e-a735-7c019f6b0b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:46.237665 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244c82e-2f55-4a9e-a735-7c019f6b0b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:46.239435 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244c82e-2f55-4a9e-a735-7c019f6b0b20 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:46.251498 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1085/2163] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:46.255652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283b94c-97fd-44fb-ac57-789807cf1a08 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:46.266554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283b94c-97fd-44fb-ac57-789807cf1a08 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:46.271874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440a011-8590-4511-ab9d-df5631842f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:46.272757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440a011-8590-4511-ab9d-df5631842f9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:46.273041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440a011-8590-4511-ab9d-df5631842f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:46.273319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440a011-8590-4511-ab9d-df5631842f9a admin admin] PATH_INFO: `/v3/projects/ef2496096b374f399606bff9a08b65eb/users/6d09fe912b4c4cab81b00df801f69680/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:46.276066 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:46.291513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e440a011-8590-4511-ab9d-df5631842f9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2496096b374f399606bff9a08b65eb, user_id=6d09fe912b4c4cab81b00df801f69680, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:46.294983 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e440a011-8590-4511-ab9d-df5631842f9a admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:46.310219 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1079/2164] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:46 2022] PUT /identity/v3/projects/ef2496096b374f399606bff9a08b65eb/users/6d09fe912b4c4cab81b00df801f69680/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:46.315728 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c9c79-e58c-4066-8375-ac4d13a61ba6 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:46.316063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c9c79-e58c-4066-8375-ac4d13a61ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:46.316398 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c9c79-e58c-4066-8375-ac4d13a61ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:46.334673 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-738c9c79-e58c-4066-8375-ac4d13a61ba6 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:46.335912 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-738c9c79-e58c-4066-8375-ac4d13a61ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:46.433209 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1086/2165] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:46.443815 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440a011-8590-4511-ab9d-df5631842f9a admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:46.445741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-e440a011-8590-4511-ab9d-df5631842f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:46.518668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440a011-8590-4511-ab9d-df5631842f9a admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:46.523072 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151ab27-92da-44eb-990c-397becbe4aa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:46.523713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151ab27-92da-44eb-990c-397becbe4aa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:46.524026 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151ab27-92da-44eb-990c-397becbe4aa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:46.524506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151ab27-92da-44eb-990c-397becbe4aa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:46.535099 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151ab27-92da-44eb-990c-397becbe4aa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:46.537853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151ab27-92da-44eb-990c-397becbe4aa5 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:46.550735 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1080/2166] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:39:51.003925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d445c0-ad78-471d-9b91-cdbc2a6c6c1e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:51.004522 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d445c0-ad78-471d-9b91-cdbc2a6c6c1e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:51.004522 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d445c0-ad78-471d-9b91-cdbc2a6c6c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:51.034861 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-d8d445c0-ad78-471d-9b91-cdbc2a6c6c1e None None] MFA Rules not processed for user `6d09fe912b4c4cab81b00df801f69680`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:51.036114 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d8d445c0-ad78-471d-9b91-cdbc2a6c6c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:51.072252 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:51.115169 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1087/2167] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:39:51.120860 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151ab27-92da-44eb-990c-397becbe4aa5 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:51.132848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151ab27-92da-44eb-990c-397becbe4aa5 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:51.138766 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62febf7c-dc15-416f-a8f9-ee4b8a4de45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:51.140962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62febf7c-dc15-416f-a8f9-ee4b8a4de45b admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:51.141254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62febf7c-dc15-416f-a8f9-ee4b8a4de45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:51.141527 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62febf7c-dc15-416f-a8f9-ee4b8a4de45b admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:51.142231 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62febf7c-dc15-416f-a8f9-ee4b8a4de45b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:51.144205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62febf7c-dc15-416f-a8f9-ee4b8a4de45b admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:51.186626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1081/2168] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:39:51 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:51.189557 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d445c0-ad78-471d-9b91-cdbc2a6c6c1e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:51.205313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d445c0-ad78-471d-9b91-cdbc2a6c6c1e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:51.296769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b91df2-9028-463a-bb9e-a05f8d7e8e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:51.297470 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b91df2-9028-463a-bb9e-a05f8d7e8e69 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:51.297754 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b91df2-9028-463a-bb9e-a05f8d7e8e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:51.298026 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b91df2-9028-463a-bb9e-a05f8d7e8e69 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:51.298720 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b91df2-9028-463a-bb9e-a05f8d7e8e69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:51.304616 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b91df2-9028-463a-bb9e-a05f8d7e8e69 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:51.332387 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1088/2169] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:39:51 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:51.336672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62febf7c-dc15-416f-a8f9-ee4b8a4de45b admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:51.349085 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62febf7c-dc15-416f-a8f9-ee4b8a4de45b admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:51.356082 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073b814-ae6a-40b7-b1fc-8e9d3abdaf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:51.356891 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073b814-ae6a-40b7-b1fc-8e9d3abdaf05 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:51.357276 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073b814-ae6a-40b7-b1fc-8e9d3abdaf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:51.357721 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073b814-ae6a-40b7-b1fc-8e9d3abdaf05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:51.358616 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a073b814-ae6a-40b7-b1fc-8e9d3abdaf05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:51.360311 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a073b814-ae6a-40b7-b1fc-8e9d3abdaf05 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:51.378382 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1082/2170] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:51.381364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b91df2-9028-463a-bb9e-a05f8d7e8e69 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:51.396707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b91df2-9028-463a-bb9e-a05f8d7e8e69 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:51.402884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0224c3a-5bf3-4771-8e74-6fdbc2bcab28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:51.403808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0224c3a-5bf3-4771-8e74-6fdbc2bcab28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:51.404150 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0224c3a-5bf3-4771-8e74-6fdbc2bcab28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:51.404418 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0224c3a-5bf3-4771-8e74-6fdbc2bcab28 admin admin] PATH_INFO: `/v3/projects/1679763b60fd4d1fb461e9881df094c0/users/35e8344f1cd44395a4b90b273d19259b/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:51.423178 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0224c3a-5bf3-4771-8e74-6fdbc2bcab28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1679763b60fd4d1fb461e9881df094c0, user_id=35e8344f1cd44395a4b90b273d19259b, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:51.426454 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0224c3a-5bf3-4771-8e74-6fdbc2bcab28 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:51.441182 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1089/2171] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:51 2022] PUT /identity/v3/projects/1679763b60fd4d1fb461e9881df094c0/users/35e8344f1cd44395a4b90b273d19259b/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:51.445786 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073b814-ae6a-40b7-b1fc-8e9d3abdaf05 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:51.460037 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073b814-ae6a-40b7-b1fc-8e9d3abdaf05 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:51.536264 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fd3ca-c278-4418-a758-7facaa569dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:51.536938 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fd3ca-c278-4418-a758-7facaa569dac admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:51.537217 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fd3ca-c278-4418-a758-7facaa569dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:51.537491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fd3ca-c278-4418-a758-7facaa569dac admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:51.538228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456fd3ca-c278-4418-a758-7facaa569dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:51.539830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456fd3ca-c278-4418-a758-7facaa569dac admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:51.551835 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1083/2172] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:51.557626 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0224c3a-5bf3-4771-8e74-6fdbc2bcab28 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:51.570518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0224c3a-5bf3-4771-8e74-6fdbc2bcab28 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:51.582810 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c148116-2e2c-43c8-999e-0cd6fb5bb2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:51.584026 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c148116-2e2c-43c8-999e-0cd6fb5bb2db admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:51.584337 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c148116-2e2c-43c8-999e-0cd6fb5bb2db admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:51.584618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c148116-2e2c-43c8-999e-0cd6fb5bb2db admin admin] PATH_INFO: `/v3/projects/1679763b60fd4d1fb461e9881df094c0/users/35e8344f1cd44395a4b90b273d19259b/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:51.586820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c148116-2e2c-43c8-999e-0cd6fb5bb2db admin admin] RBAC: Authorizing `identity:create_grant(project_id=1679763b60fd4d1fb461e9881df094c0, user_id=35e8344f1cd44395a4b90b273d19259b, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:51.590264 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c148116-2e2c-43c8-999e-0cd6fb5bb2db admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:51.601649 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1090/2173] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:51 2022] PUT /identity/v3/projects/1679763b60fd4d1fb461e9881df094c0/users/35e8344f1cd44395a4b90b273d19259b/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:53.157187 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fd3ca-c278-4418-a758-7facaa569dac admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:53.158947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-456fd3ca-c278-4418-a758-7facaa569dac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:53.192684 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fd3ca-c278-4418-a758-7facaa569dac admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:53.301165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:53.319520 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7de56f-8a26-45d1-a468-55279c4de4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:53.320321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7de56f-8a26-45d1-a468-55279c4de4ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:53.322630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7de56f-8a26-45d1-a468-55279c4de4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:53.323179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7de56f-8a26-45d1-a468-55279c4de4ec admin admin] PATH_INFO: `/v3/users/0fad66f1434a44bb9a22a78bf3a8a3ab` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:53.325073 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7de56f-8a26-45d1-a468-55279c4de4ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fad66f1434a44bb9a22a78bf3a8a3ab)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:53.328806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7de56f-8a26-45d1-a468-55279c4de4ec admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:53.436208 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1c7de56f-8a26-45d1-a468-55279c4de4ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fad66f1434a44bb9a22a78bf3a8a3ab'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:53.455140 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1c7de56f-8a26-45d1-a468-55279c4de4ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fad66f1434a44bb9a22a78bf3a8a3ab'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:53.465935 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1c7de56f-8a26-45d1-a468-55279c4de4ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fad66f1434a44bb9a22a78bf3a8a3ab'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:53.467495 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1c7de56f-8a26-45d1-a468-55279c4de4ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fad66f1434a44bb9a22a78bf3a8a3ab'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:53.489864 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1c7de56f-8a26-45d1-a468-55279c4de4ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fad66f1434a44bb9a22a78bf3a8a3ab'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:53.505739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1084/2174] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:39:53 2022] DELETE /identity/v3/users/0fad66f1434a44bb9a22a78bf3a8a3ab => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:53.512424 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906e48a-7e7c-4b31-bcd4-fafb9ed4ce25 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:53.512992 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906e48a-7e7c-4b31-bcd4-fafb9ed4ce25 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:53.513194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906e48a-7e7c-4b31-bcd4-fafb9ed4ce25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:53.546770 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-e906e48a-7e7c-4b31-bcd4-fafb9ed4ce25 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:53.548518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e906e48a-7e7c-4b31-bcd4-fafb9ed4ce25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:53.706674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1091/2175] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:53.718697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7de56f-8a26-45d1-a468-55279c4de4ec admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:53.720246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-1c7de56f-8a26-45d1-a468-55279c4de4ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:53.859013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7de56f-8a26-45d1-a468-55279c4de4ec admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:53.866291 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483c816-5466-4a2e-ab7b-abd5e5dd3643 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:53.867277 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483c816-5466-4a2e-ab7b-abd5e5dd3643 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:53.867752 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483c816-5466-4a2e-ab7b-abd5e5dd3643 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:53.868247 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483c816-5466-4a2e-ab7b-abd5e5dd3643 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:53.885918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6483c816-5466-4a2e-ab7b-abd5e5dd3643 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:53.888614 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6483c816-5466-4a2e-ab7b-abd5e5dd3643 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:53.916746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1085/2176] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:39:54.272589 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906e48a-7e7c-4b31-bcd4-fafb9ed4ce25 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:54.274062 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e906e48a-7e7c-4b31-bcd4-fafb9ed4ce25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:54.297334 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906e48a-7e7c-4b31-bcd4-fafb9ed4ce25 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:54.318524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bb467-f2fc-4648-a12b-676f070a9169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:54.319355 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bb467-f2fc-4648-a12b-676f070a9169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:54.319632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bb467-f2fc-4648-a12b-676f070a9169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:54.319957 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bb467-f2fc-4648-a12b-676f070a9169 admin admin] PATH_INFO: `/v3/projects/5a29ea01c9a24cd182875abfde0eef67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:54.321392 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898bb467-f2fc-4648-a12b-676f070a9169 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a29ea01c9a24cd182875abfde0eef67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:54.324989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898bb467-f2fc-4648-a12b-676f070a9169 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:54.366273 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:54.430688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-898bb467-f2fc-4648-a12b-676f070a9169 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a29ea01c9a24cd182875abfde0eef67', 'request_id': 'req-898bb467-f2fc-4648-a12b-676f070a9169'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:54.444735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-898bb467-f2fc-4648-a12b-676f070a9169 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a29ea01c9a24cd182875abfde0eef67', 'request_id': 'req-898bb467-f2fc-4648-a12b-676f070a9169'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:54.457355 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-898bb467-f2fc-4648-a12b-676f070a9169 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a29ea01c9a24cd182875abfde0eef67', 'request_id': 'req-898bb467-f2fc-4648-a12b-676f070a9169'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:54.477133 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-898bb467-f2fc-4648-a12b-676f070a9169 admin admin] The token cache is being invalidate because project 5a29ea01c9a24cd182875abfde0eef67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:39:54.477370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-898bb467-f2fc-4648-a12b-676f070a9169 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:54.478127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-898bb467-f2fc-4648-a12b-676f070a9169 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:54.483333 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1092/2177] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:39:54 2022] DELETE /identity/v3/projects/5a29ea01c9a24cd182875abfde0eef67 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:54.485134 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483c816-5466-4a2e-ab7b-abd5e5dd3643 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:54.507078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483c816-5466-4a2e-ab7b-abd5e5dd3643 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:54.620694 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20d554-9a8d-4414-af5d-32d0ce937845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:54.621612 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20d554-9a8d-4414-af5d-32d0ce937845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:54.621990 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20d554-9a8d-4414-af5d-32d0ce937845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:54.622294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20d554-9a8d-4414-af5d-32d0ce937845 admin admin] PATH_INFO: `/v3/users/af9ac825c8b54daa8e6a5aa9c0592471` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:54.623646 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db20d554-9a8d-4414-af5d-32d0ce937845 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af9ac825c8b54daa8e6a5aa9c0592471)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:54.626399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db20d554-9a8d-4414-af5d-32d0ce937845 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:54.645272 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:54.741524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-db20d554-9a8d-4414-af5d-32d0ce937845 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af9ac825c8b54daa8e6a5aa9c0592471'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:54.760386 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-db20d554-9a8d-4414-af5d-32d0ce937845 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af9ac825c8b54daa8e6a5aa9c0592471'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:54.784561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-db20d554-9a8d-4414-af5d-32d0ce937845 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af9ac825c8b54daa8e6a5aa9c0592471'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:54.785704 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-db20d554-9a8d-4414-af5d-32d0ce937845 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af9ac825c8b54daa8e6a5aa9c0592471'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:54.811096 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-db20d554-9a8d-4414-af5d-32d0ce937845 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af9ac825c8b54daa8e6a5aa9c0592471'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:54.819485 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1086/2178] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:39:54 2022] DELETE /identity/v3/users/af9ac825c8b54daa8e6a5aa9c0592471 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:39:55.628346 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bb467-f2fc-4648-a12b-676f070a9169 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:55.630055 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-898bb467-f2fc-4648-a12b-676f070a9169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:55.655355 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bb467-f2fc-4648-a12b-676f070a9169 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:55.778560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273bd1f5-2c30-42d6-8ad9-6d22ffa43c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:55.779618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bd1f5-2c30-42d6-8ad9-6d22ffa43c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:55.780081 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bd1f5-2c30-42d6-8ad9-6d22ffa43c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:55.780706 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bd1f5-2c30-42d6-8ad9-6d22ffa43c8e admin admin] PATH_INFO: `/v3/projects/cce458aea2784ea895855be4c1fce6d1` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:55.784578 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273bd1f5-2c30-42d6-8ad9-6d22ffa43c8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cce458aea2784ea895855be4c1fce6d1)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:55.787195 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273bd1f5-2c30-42d6-8ad9-6d22ffa43c8e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:55.858319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-273bd1f5-2c30-42d6-8ad9-6d22ffa43c8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cce458aea2784ea895855be4c1fce6d1', 'request_id': 'req-273bd1f5-2c30-42d6-8ad9-6d22ffa43c8e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:55.872671 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-273bd1f5-2c30-42d6-8ad9-6d22ffa43c8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cce458aea2784ea895855be4c1fce6d1', 'request_id': 'req-273bd1f5-2c30-42d6-8ad9-6d22ffa43c8e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:55.881475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-273bd1f5-2c30-42d6-8ad9-6d22ffa43c8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cce458aea2784ea895855be4c1fce6d1', 'request_id': 'req-273bd1f5-2c30-42d6-8ad9-6d22ffa43c8e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:55.896090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-273bd1f5-2c30-42d6-8ad9-6d22ffa43c8e admin admin] The token cache is being invalidate because project cce458aea2784ea895855be4c1fce6d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:39:55.896697 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-273bd1f5-2c30-42d6-8ad9-6d22ffa43c8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:55.897825 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-273bd1f5-2c30-42d6-8ad9-6d22ffa43c8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:39:55.902612 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1093/2179] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:39:55 2022] DELETE /identity/v3/projects/cce458aea2784ea895855be4c1fce6d1 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:55.907174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6385dcc-b73f-4df6-8d70-735da678fb2b None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:55.907597 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6385dcc-b73f-4df6-8d70-735da678fb2b None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:55.907897 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6385dcc-b73f-4df6-8d70-735da678fb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:55.940053 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-a6385dcc-b73f-4df6-8d70-735da678fb2b None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:55.941278 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a6385dcc-b73f-4df6-8d70-735da678fb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:56.062300 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1087/2180] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:56.071354 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5e3d1-e52a-40fd-ad72-a85d6f1bc4d1 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:56.071649 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5e3d1-e52a-40fd-ad72-a85d6f1bc4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:56.072194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5e3d1-e52a-40fd-ad72-a85d6f1bc4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:56.111935 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-78a5e3d1-e52a-40fd-ad72-a85d6f1bc4d1 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:56.113380 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-78a5e3d1-e52a-40fd-ad72-a85d6f1bc4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:56.129071 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1094/2181] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:56.133727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6385dcc-b73f-4df6-8d70-735da678fb2b None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:56.149088 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6385dcc-b73f-4df6-8d70-735da678fb2b None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:56.158127 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eaab2a-dd9b-4be5-b197-60cccdd80d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:56.159053 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eaab2a-dd9b-4be5-b197-60cccdd80d83 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:56.159352 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eaab2a-dd9b-4be5-b197-60cccdd80d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:56.159723 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eaab2a-dd9b-4be5-b197-60cccdd80d83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:56.160729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eaab2a-dd9b-4be5-b197-60cccdd80d83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:39:56.161073 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eaab2a-dd9b-4be5-b197-60cccdd80d83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:56.162726 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eaab2a-dd9b-4be5-b197-60cccdd80d83 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:56.182760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1088/2182] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:39:56.183751 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5e3d1-e52a-40fd-ad72-a85d6f1bc4d1 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:56.199816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5e3d1-e52a-40fd-ad72-a85d6f1bc4d1 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:56.213444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec50efb-aa51-412f-a532-63510a92d6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:56.214142 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec50efb-aa51-412f-a532-63510a92d6e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:56.214431 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec50efb-aa51-412f-a532-63510a92d6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:56.215081 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec50efb-aa51-412f-a532-63510a92d6e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:56.215409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec50efb-aa51-412f-a532-63510a92d6e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:56.218724 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec50efb-aa51-412f-a532-63510a92d6e6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:56.267682 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1095/2183] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:39:56 2022] POST /identity/v3/projects => generated 365 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:56.268789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eaab2a-dd9b-4be5-b197-60cccdd80d83 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:56.289745 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eaab2a-dd9b-4be5-b197-60cccdd80d83 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:56.541719 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6950b-2665-4c74-9ee6-f84950a05b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:56.542450 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6950b-2665-4c74-9ee6-f84950a05b9c admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:56.542760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6950b-2665-4c74-9ee6-f84950a05b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:56.543042 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6950b-2665-4c74-9ee6-f84950a05b9c admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:56.543741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6950b-2665-4c74-9ee6-f84950a05b9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:56.550286 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6950b-2665-4c74-9ee6-f84950a05b9c admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:56.647061 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1089/2184] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:39:56 2022] POST /identity/v3/users => generated 283 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:56.651804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec50efb-aa51-412f-a532-63510a92d6e6 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:56.691522 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec50efb-aa51-412f-a532-63510a92d6e6 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:56.712433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fc493-0e9a-4dba-bfc9-209aee3c385d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:56.715865 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fc493-0e9a-4dba-bfc9-209aee3c385d admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:56.717113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fc493-0e9a-4dba-bfc9-209aee3c385d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:56.718431 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fc493-0e9a-4dba-bfc9-209aee3c385d admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:56.720615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8fc493-0e9a-4dba-bfc9-209aee3c385d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:56.724877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8fc493-0e9a-4dba-bfc9-209aee3c385d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:56.731625 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:56.764104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1096/2185] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:56.765922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6950b-2665-4c74-9ee6-f84950a05b9c admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:56.783257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6950b-2665-4c74-9ee6-f84950a05b9c admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:56.789471 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc7ab3-8ff5-46bb-b5d0-2e279b0d3d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:56.790310 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc7ab3-8ff5-46bb-b5d0-2e279b0d3d29 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:56.792060 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc7ab3-8ff5-46bb-b5d0-2e279b0d3d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:56.792060 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc7ab3-8ff5-46bb-b5d0-2e279b0d3d29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:56.793134 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc7ab3-8ff5-46bb-b5d0-2e279b0d3d29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:56.794976 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc7ab3-8ff5-46bb-b5d0-2e279b0d3d29 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:56.797602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:56.864932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fc493-0e9a-4dba-bfc9-209aee3c385d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:56.865532 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1090/2186] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:56.880115 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fc493-0e9a-4dba-bfc9-209aee3c385d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:56.903180 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca0727-aea5-49c6-8eb6-1e062d9e3606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:56.906627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca0727-aea5-49c6-8eb6-1e062d9e3606 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:56.906788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca0727-aea5-49c6-8eb6-1e062d9e3606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:56.907005 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca0727-aea5-49c6-8eb6-1e062d9e3606 admin admin] PATH_INFO: `/v3/projects/ea93f3760a804415a48484d85e5d841e/users/82177dfeb4844c2dad46ce59dd4544f4/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:56.953327 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca0727-aea5-49c6-8eb6-1e062d9e3606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea93f3760a804415a48484d85e5d841e, user_id=82177dfeb4844c2dad46ce59dd4544f4, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:56.957670 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca0727-aea5-49c6-8eb6-1e062d9e3606 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:57.018880 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1097/2187] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:56 2022] PUT /identity/v3/projects/ea93f3760a804415a48484d85e5d841e/users/82177dfeb4844c2dad46ce59dd4544f4/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:39:57.022388 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1b0d5-97b6-4e11-b2b5-d257ad75494b None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:57.023467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1b0d5-97b6-4e11-b2b5-d257ad75494b None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:57.023969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1b0d5-97b6-4e11-b2b5-d257ad75494b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:57.097337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-e3f1b0d5-97b6-4e11-b2b5-d257ad75494b None None] MFA Rules not processed for user `82177dfeb4844c2dad46ce59dd4544f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:57.099275 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-e3f1b0d5-97b6-4e11-b2b5-d257ad75494b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:57.258089 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1091/2188] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:57 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:57.262692 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca0727-aea5-49c6-8eb6-1e062d9e3606 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:57.381669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca0727-aea5-49c6-8eb6-1e062d9e3606 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:57.389745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b85ee-d900-4db1-b10d-cbaaba45e8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:57.396779 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b85ee-d900-4db1-b10d-cbaaba45e8a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:57.403818 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b85ee-d900-4db1-b10d-cbaaba45e8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:57.403818 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b85ee-d900-4db1-b10d-cbaaba45e8a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:57.403818 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b85ee-d900-4db1-b10d-cbaaba45e8a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:57.403818 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b85ee-d900-4db1-b10d-cbaaba45e8a3 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:57.514271 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1b0d5-97b6-4e11-b2b5-d257ad75494b None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:57.524004 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1098/2189] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:39:57 2022] POST /identity/v3/projects => generated 367 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:57.543535 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1b0d5-97b6-4e11-b2b5-d257ad75494b None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:57.690596 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194d27a-96fb-4426-a9bb-aafde2d3717d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:57.691345 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194d27a-96fb-4426-a9bb-aafde2d3717d admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:57.691675 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194d27a-96fb-4426-a9bb-aafde2d3717d admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:57.691998 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194d27a-96fb-4426-a9bb-aafde2d3717d admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:57.692808 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194d27a-96fb-4426-a9bb-aafde2d3717d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:57.694873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194d27a-96fb-4426-a9bb-aafde2d3717d admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:57.740325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1092/2190] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:39:57 2022] POST /identity/v3/users => generated 290 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:39:57.744602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b85ee-d900-4db1-b10d-cbaaba45e8a3 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:57.766615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b85ee-d900-4db1-b10d-cbaaba45e8a3 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:57.784516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d052803-8e39-49cf-9c50-d32974d7fed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:57.785465 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d052803-8e39-49cf-9c50-d32974d7fed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:57.786097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d052803-8e39-49cf-9c50-d32974d7fed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:57.786705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d052803-8e39-49cf-9c50-d32974d7fed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:57.787757 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d052803-8e39-49cf-9c50-d32974d7fed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:57.789747 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d052803-8e39-49cf-9c50-d32974d7fed7 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:57.808256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194d27a-96fb-4426-a9bb-aafde2d3717d admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:57.820241 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1099/2191] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:57.828040 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194d27a-96fb-4426-a9bb-aafde2d3717d admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:57.848637 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2b381-fbb5-429a-85f7-6ba407f71792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:57.850604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2b381-fbb5-429a-85f7-6ba407f71792 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:57.850899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2b381-fbb5-429a-85f7-6ba407f71792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:57.851185 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2b381-fbb5-429a-85f7-6ba407f71792 admin admin] PATH_INFO: `/v3/projects/2e11b6eb820642d48a22afd352d8a9dd/users/1198aa29640e4b10a6dc2b34458197af/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:57.911284 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f2b381-fbb5-429a-85f7-6ba407f71792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e11b6eb820642d48a22afd352d8a9dd, user_id=1198aa29640e4b10a6dc2b34458197af, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:57.914771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f2b381-fbb5-429a-85f7-6ba407f71792 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:57.951979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1093/2192] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:57 2022] PUT /identity/v3/projects/2e11b6eb820642d48a22afd352d8a9dd/users/1198aa29640e4b10a6dc2b34458197af/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:39:57.956648 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d052803-8e39-49cf-9c50-d32974d7fed7 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:57.971422 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d052803-8e39-49cf-9c50-d32974d7fed7 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:58.153292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ac364-c754-4798-9cef-4187bef37697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:58.158289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ac364-c754-4798-9cef-4187bef37697 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:58.158589 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ac364-c754-4798-9cef-4187bef37697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:58.158878 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ac364-c754-4798-9cef-4187bef37697 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:58.159729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ac364-c754-4798-9cef-4187bef37697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:58.161555 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ac364-c754-4798-9cef-4187bef37697 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:58.212061 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2b381-fbb5-429a-85f7-6ba407f71792 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:58.212648 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1100/2193] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:39:58.231377 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2b381-fbb5-429a-85f7-6ba407f71792 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:58.240207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc373ce-ef4e-47e7-bfe2-69a550ffbfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:58.241208 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc373ce-ef4e-47e7-bfe2-69a550ffbfb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:58.241530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc373ce-ef4e-47e7-bfe2-69a550ffbfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:58.241814 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc373ce-ef4e-47e7-bfe2-69a550ffbfb4 admin admin] PATH_INFO: `/v3/projects/2e11b6eb820642d48a22afd352d8a9dd/users/1198aa29640e4b10a6dc2b34458197af/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:58.247057 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc373ce-ef4e-47e7-bfe2-69a550ffbfb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e11b6eb820642d48a22afd352d8a9dd, user_id=1198aa29640e4b10a6dc2b34458197af, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:58.250105 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc373ce-ef4e-47e7-bfe2-69a550ffbfb4 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:58.283350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1094/2194] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:39:58 2022] PUT /identity/v3/projects/2e11b6eb820642d48a22afd352d8a9dd/users/1198aa29640e4b10a6dc2b34458197af/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:39:58.297610 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687afa79-bf90-4c67-96d2-014f83653979 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:58.298419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687afa79-bf90-4c67-96d2-014f83653979 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:58.299114 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687afa79-bf90-4c67-96d2-014f83653979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:58.362856 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-687afa79-bf90-4c67-96d2-014f83653979 None None] MFA Rules not processed for user `1198aa29640e4b10a6dc2b34458197af`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:58.365134 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-687afa79-bf90-4c67-96d2-014f83653979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:58.455904 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5ede6-009f-40ed-8fab-59096e1cb3dd None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:58.456445 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5ede6-009f-40ed-8fab-59096e1cb3dd None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:58.457136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5ede6-009f-40ed-8fab-59096e1cb3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:58.481355 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:58.533967 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-2ee5ede6-009f-40ed-8fab-59096e1cb3dd None None] MFA Rules not processed for user `35e8344f1cd44395a4b90b273d19259b`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:39:58.540098 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-2ee5ede6-009f-40ed-8fab-59096e1cb3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:58.564657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1101/2195] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:58 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:58.576590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687afa79-bf90-4c67-96d2-014f83653979 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:58.579340 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-687afa79-bf90-4c67-96d2-014f83653979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:58.591183 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:39:58.671311 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1095/2196] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:39:58 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:39:58.689972 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5ede6-009f-40ed-8fab-59096e1cb3dd None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:58.691660 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-2ee5ede6-009f-40ed-8fab-59096e1cb3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:58.713681 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687afa79-bf90-4c67-96d2-014f83653979 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:58.737494 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbbc86-2e26-4e43-bc5e-d0b9ecb763b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:58.738627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbbc86-2e26-4e43-bc5e-d0b9ecb763b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:58.738913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbbc86-2e26-4e43-bc5e-d0b9ecb763b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:58.739267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbbc86-2e26-4e43-bc5e-d0b9ecb763b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:58.759219 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbbc86-2e26-4e43-bc5e-d0b9ecb763b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:58.772861 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbbc86-2e26-4e43-bc5e-d0b9ecb763b9 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:58.846233 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5ede6-009f-40ed-8fab-59096e1cb3dd None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:58.852170 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1102/2197] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:58 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:39:58.862835 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7bdbd-7412-464b-849e-8bcfa51a594a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:58.863499 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7bdbd-7412-464b-849e-8bcfa51a594a service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:58.863814 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7bdbd-7412-464b-849e-8bcfa51a594a service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:58.864079 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7bdbd-7412-464b-849e-8bcfa51a594a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:58.869282 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-2ac7bdbd-7412-464b-849e-8bcfa51a594a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:39:58.982192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbbc86-2e26-4e43-bc5e-d0b9ecb763b9 service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:39:58.998025 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbbc86-2e26-4e43-bc5e-d0b9ecb763b9 service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:39:59.011588 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5fe70-96da-4a1f-98d1-63a3c9397eeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:39:59.012431 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5fe70-96da-4a1f-98d1-63a3c9397eeb service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:39:59.012569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5fe70-96da-4a1f-98d1-63a3c9397eeb service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:39:59.012841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5fe70-96da-4a1f-98d1-63a3c9397eeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:39:59.028167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac7bdbd-7412-464b-849e-8bcfa51a594a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:59.030002 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac7bdbd-7412-464b-849e-8bcfa51a594a service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:59.060872 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1096/2198] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:39:59.116192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5fe70-96da-4a1f-98d1-63a3c9397eeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:39:59.129034 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5fe70-96da-4a1f-98d1-63a3c9397eeb service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:39:59.146487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1103/2199] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:39:58 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:40:00.530792 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4baf55-1dac-42cd-a381-ebddbb7eef60 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:00.531271 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4baf55-1dac-42cd-a381-ebddbb7eef60 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:00.531371 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4baf55-1dac-42cd-a381-ebddbb7eef60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:00.548960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-6f4baf55-1dac-42cd-a381-ebddbb7eef60 None None] MFA Rules not processed for user `6d09fe912b4c4cab81b00df801f69680`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:00.550403 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-6f4baf55-1dac-42cd-a381-ebddbb7eef60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:00.558337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1097/2200] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:07.667230 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5fe70-96da-4a1f-98d1-63a3c9397eeb service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:07.735082 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5fe70-96da-4a1f-98d1-63a3c9397eeb service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:07.740810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4baf55-1dac-42cd-a381-ebddbb7eef60 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:07.742135 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aeb0bd-c7b2-484a-8b7d-65d196558a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:07.743023 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aeb0bd-c7b2-484a-8b7d-65d196558a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:07.743404 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aeb0bd-c7b2-484a-8b7d-65d196558a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:07.743689 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aeb0bd-c7b2-484a-8b7d-65d196558a0e admin admin] PATH_INFO: `/v3/users/82177dfeb4844c2dad46ce59dd4544f4` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:07.744183 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-6f4baf55-1dac-42cd-a381-ebddbb7eef60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:07.745206 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aeb0bd-c7b2-484a-8b7d-65d196558a0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=82177dfeb4844c2dad46ce59dd4544f4)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:07.747171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aeb0bd-c7b2-484a-8b7d-65d196558a0e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:07.760361 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4baf55-1dac-42cd-a381-ebddbb7eef60 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:07.761524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:07.765033 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ccf8b7-5ddd-4df3-9a3a-97ece6cbe200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:07.777987 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccf8b7-5ddd-4df3-9a3a-97ece6cbe200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:07.778279 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccf8b7-5ddd-4df3-9a3a-97ece6cbe200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:07.780654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccf8b7-5ddd-4df3-9a3a-97ece6cbe200 admin admin] PATH_INFO: `/v3/users/061ea711fcb9428386e6bf7344146ae5` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:07.782586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ccf8b7-5ddd-4df3-9a3a-97ece6cbe200 admin admin] RBAC: Authorizing `identity:delete_user(user_id=061ea711fcb9428386e6bf7344146ae5)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:07.788754 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ccf8b7-5ddd-4df3-9a3a-97ece6cbe200 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:07.830958 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-57aeb0bd-c7b2-484a-8b7d-65d196558a0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82177dfeb4844c2dad46ce59dd4544f4'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:07.839197 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-57aeb0bd-c7b2-484a-8b7d-65d196558a0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82177dfeb4844c2dad46ce59dd4544f4'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:07.839965 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-57aeb0bd-c7b2-484a-8b7d-65d196558a0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82177dfeb4844c2dad46ce59dd4544f4'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:07.840619 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-57aeb0bd-c7b2-484a-8b7d-65d196558a0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82177dfeb4844c2dad46ce59dd4544f4'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:07.848415 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-69ccf8b7-5ddd-4df3-9a3a-97ece6cbe200 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '061ea711fcb9428386e6bf7344146ae5'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:07.850037 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:07.853376 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-57aeb0bd-c7b2-484a-8b7d-65d196558a0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82177dfeb4844c2dad46ce59dd4544f4'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:07.857633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-69ccf8b7-5ddd-4df3-9a3a-97ece6cbe200 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '061ea711fcb9428386e6bf7344146ae5'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:07.864614 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-69ccf8b7-5ddd-4df3-9a3a-97ece6cbe200 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '061ea711fcb9428386e6bf7344146ae5'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:07.865257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-69ccf8b7-5ddd-4df3-9a3a-97ece6cbe200 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '061ea711fcb9428386e6bf7344146ae5'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:07.866909 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1104/2201] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:40:07 2022] DELETE /identity/v3/users/82177dfeb4844c2dad46ce59dd4544f4 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:07.871466 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd29ca-cb52-4407-b45f-06a06f4ab6c5 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:07.871755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd29ca-cb52-4407-b45f-06a06f4ab6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:07.872076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd29ca-cb52-4407-b45f-06a06f4ab6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:07.880145 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-69ccf8b7-5ddd-4df3-9a3a-97ece6cbe200 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '061ea711fcb9428386e6bf7344146ae5'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:07.887045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1098/2202] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:40:07 2022] DELETE /identity/v3/users/061ea711fcb9428386e6bf7344146ae5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:07.890629 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-7cbd29ca-cb52-4407-b45f-06a06f4ab6c5 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:07.891400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494eba88-1e49-4f15-a03e-a9002a330786 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:07.891714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494eba88-1e49-4f15-a03e-a9002a330786 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:07.891913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-7cbd29ca-cb52-4407-b45f-06a06f4ab6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:07.892048 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494eba88-1e49-4f15-a03e-a9002a330786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:07.915859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-494eba88-1e49-4f15-a03e-a9002a330786 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:07.917890 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-494eba88-1e49-4f15-a03e-a9002a330786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:07.958578 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1105/2203] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:40:07.971909 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd29ca-cb52-4407-b45f-06a06f4ab6c5 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:07.974407 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-7cbd29ca-cb52-4407-b45f-06a06f4ab6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:07.999890 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1099/2204] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:08.011103 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494eba88-1e49-4f15-a03e-a9002a330786 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:08.013696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-494eba88-1e49-4f15-a03e-a9002a330786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:08.046556 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd29ca-cb52-4407-b45f-06a06f4ab6c5 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:08.050747 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d37ce8-0350-41b2-af15-83b07c1155bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:08.051178 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d37ce8-0350-41b2-af15-83b07c1155bf service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:08.051578 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d37ce8-0350-41b2-af15-83b07c1155bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:08.051883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d37ce8-0350-41b2-af15-83b07c1155bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:08.066073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d37ce8-0350-41b2-af15-83b07c1155bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:08.068044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d37ce8-0350-41b2-af15-83b07c1155bf service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:08.080005 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1106/2205] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:40:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:40:08.098698 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494eba88-1e49-4f15-a03e-a9002a330786 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:08.106203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ec5cf-c69c-412d-964b-3cf295943e70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:08.106886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ec5cf-c69c-412d-964b-3cf295943e70 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:08.107523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ec5cf-c69c-412d-964b-3cf295943e70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:08.107523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ec5cf-c69c-412d-964b-3cf295943e70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:08.125265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ec5cf-c69c-412d-964b-3cf295943e70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:08.127154 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ec5cf-c69c-412d-964b-3cf295943e70 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:08.148610 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1100/2206] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:40:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:40:08.650788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d37ce8-0350-41b2-af15-83b07c1155bf service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:08.652492 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ec5cf-c69c-412d-964b-3cf295943e70 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:08.653828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-39d37ce8-0350-41b2-af15-83b07c1155bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:08.655632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-7a9ec5cf-c69c-412d-964b-3cf295943e70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:08.682517 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d37ce8-0350-41b2-af15-83b07c1155bf service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:08.684037 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ec5cf-c69c-412d-964b-3cf295943e70 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:08.690502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645f747-1dc7-43db-a893-97a9899f1e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:08.690869 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d8f13-0324-46fe-ac7a-b93b982c0b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:08.692102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d8f13-0324-46fe-ac7a-b93b982c0b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:08.692222 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645f747-1dc7-43db-a893-97a9899f1e48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:08.692626 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d8f13-0324-46fe-ac7a-b93b982c0b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:08.692901 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645f747-1dc7-43db-a893-97a9899f1e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:08.693165 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d8f13-0324-46fe-ac7a-b93b982c0b56 admin admin] PATH_INFO: `/v3/projects/ea93f3760a804415a48484d85e5d841e` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:08.693471 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645f747-1dc7-43db-a893-97a9899f1e48 admin admin] PATH_INFO: `/v3/projects/ee48ac94a2414c70963263a637921502` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:08.694518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d8f13-0324-46fe-ac7a-b93b982c0b56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea93f3760a804415a48484d85e5d841e)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:08.695161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645f747-1dc7-43db-a893-97a9899f1e48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee48ac94a2414c70963263a637921502)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:08.703816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d8f13-0324-46fe-ac7a-b93b982c0b56 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:08.706029 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645f747-1dc7-43db-a893-97a9899f1e48 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:08.737285 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:08.763480 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1c1d8f13-0324-46fe-ac7a-b93b982c0b56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea93f3760a804415a48484d85e5d841e', 'request_id': 'req-1c1d8f13-0324-46fe-ac7a-b93b982c0b56'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:08.777548 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1c1d8f13-0324-46fe-ac7a-b93b982c0b56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea93f3760a804415a48484d85e5d841e', 'request_id': 'req-1c1d8f13-0324-46fe-ac7a-b93b982c0b56'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:08.789300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1c1d8f13-0324-46fe-ac7a-b93b982c0b56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea93f3760a804415a48484d85e5d841e', 'request_id': 'req-1c1d8f13-0324-46fe-ac7a-b93b982c0b56'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:08.790468 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6645f747-1dc7-43db-a893-97a9899f1e48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee48ac94a2414c70963263a637921502', 'request_id': 'req-6645f747-1dc7-43db-a893-97a9899f1e48'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:08.800019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6645f747-1dc7-43db-a893-97a9899f1e48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee48ac94a2414c70963263a637921502', 'request_id': 'req-6645f747-1dc7-43db-a893-97a9899f1e48'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:08.805252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:08.813508 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1c1d8f13-0324-46fe-ac7a-b93b982c0b56 admin admin] The token cache is being invalidate because project ea93f3760a804415a48484d85e5d841e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:40:08.813813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1c1d8f13-0324-46fe-ac7a-b93b982c0b56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:08.814527 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6645f747-1dc7-43db-a893-97a9899f1e48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee48ac94a2414c70963263a637921502', 'request_id': 'req-6645f747-1dc7-43db-a893-97a9899f1e48'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:08.815015 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1c1d8f13-0324-46fe-ac7a-b93b982c0b56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:08.816705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1107/2207] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:40:08 2022] DELETE /identity/v3/projects/ea93f3760a804415a48484d85e5d841e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:08.820543 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d8f13-0324-46fe-ac7a-b93b982c0b56 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:08.822606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1c1d8f13-0324-46fe-ac7a-b93b982c0b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:08.831026 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6645f747-1dc7-43db-a893-97a9899f1e48 admin admin] The token cache is being invalidate because project ee48ac94a2414c70963263a637921502 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:40:08.832061 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6645f747-1dc7-43db-a893-97a9899f1e48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:08.833165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6645f747-1dc7-43db-a893-97a9899f1e48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:08.835314 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1101/2208] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:40:08 2022] DELETE /identity/v3/projects/ee48ac94a2414c70963263a637921502 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:08.840652 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645f747-1dc7-43db-a893-97a9899f1e48 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:08.843000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-6645f747-1dc7-43db-a893-97a9899f1e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:08.917399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d8f13-0324-46fe-ac7a-b93b982c0b56 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:08.939322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645f747-1dc7-43db-a893-97a9899f1e48 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:08.944941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d43859-a0d3-45f4-9c3e-73c47540cbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:08.945982 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d43859-a0d3-45f4-9c3e-73c47540cbee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:08.946394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d43859-a0d3-45f4-9c3e-73c47540cbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:08.947343 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d43859-a0d3-45f4-9c3e-73c47540cbee admin admin] PATH_INFO: `/v3/users/6c10e4eeddbc43bba2fdcd377b0dbb5f` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:08.949009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d43859-a0d3-45f4-9c3e-73c47540cbee admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c10e4eeddbc43bba2fdcd377b0dbb5f)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:08.953803 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d43859-a0d3-45f4-9c3e-73c47540cbee admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:09.003319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e8177-e3a5-464b-aff5-ddb9c624f007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:09.004500 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e8177-e3a5-464b-aff5-ddb9c624f007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:09.004856 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e8177-e3a5-464b-aff5-ddb9c624f007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:09.005201 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e8177-e3a5-464b-aff5-ddb9c624f007 admin admin] PATH_INFO: `/v3/users/1198aa29640e4b10a6dc2b34458197af` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:09.007229 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e8177-e3a5-464b-aff5-ddb9c624f007 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1198aa29640e4b10a6dc2b34458197af)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:09.009952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e8177-e3a5-464b-aff5-ddb9c624f007 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:09.016149 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-88d43859-a0d3-45f4-9c3e-73c47540cbee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c10e4eeddbc43bba2fdcd377b0dbb5f'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:09.024942 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-88d43859-a0d3-45f4-9c3e-73c47540cbee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c10e4eeddbc43bba2fdcd377b0dbb5f'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:09.033705 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-88d43859-a0d3-45f4-9c3e-73c47540cbee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c10e4eeddbc43bba2fdcd377b0dbb5f'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:09.034547 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-88d43859-a0d3-45f4-9c3e-73c47540cbee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c10e4eeddbc43bba2fdcd377b0dbb5f'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:09.068885 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-88d43859-a0d3-45f4-9c3e-73c47540cbee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c10e4eeddbc43bba2fdcd377b0dbb5f'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:09.072919 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1d1e8177-e3a5-464b-aff5-ddb9c624f007 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1198aa29640e4b10a6dc2b34458197af'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:09.077525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1102/2209] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:40:08 2022] DELETE /identity/v3/users/6c10e4eeddbc43bba2fdcd377b0dbb5f => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:09.083492 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1d1e8177-e3a5-464b-aff5-ddb9c624f007 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1198aa29640e4b10a6dc2b34458197af'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:09.084856 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1d1e8177-e3a5-464b-aff5-ddb9c624f007 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1198aa29640e4b10a6dc2b34458197af'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:09.085940 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1d1e8177-e3a5-464b-aff5-ddb9c624f007 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1198aa29640e4b10a6dc2b34458197af'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:09.095036 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:09.102223 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1d1e8177-e3a5-464b-aff5-ddb9c624f007 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1198aa29640e4b10a6dc2b34458197af'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:09.122135 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1108/2210] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:40:08 2022] DELETE /identity/v3/users/1198aa29640e4b10a6dc2b34458197af => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:40:09.513221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d43859-a0d3-45f4-9c3e-73c47540cbee admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:09.514766 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-88d43859-a0d3-45f4-9c3e-73c47540cbee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:09.615414 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e8177-e3a5-464b-aff5-ddb9c624f007 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:09.617133 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1d1e8177-e3a5-464b-aff5-ddb9c624f007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:09.621247 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d43859-a0d3-45f4-9c3e-73c47540cbee admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:09.643161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:09.732029 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e8177-e3a5-464b-aff5-ddb9c624f007 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:09.734289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3cf1ef-8c23-450a-8922-49127d78c2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:09.735176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cf1ef-8c23-450a-8922-49127d78c2c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:09.735492 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cf1ef-8c23-450a-8922-49127d78c2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:09.736038 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cf1ef-8c23-450a-8922-49127d78c2c2 admin admin] PATH_INFO: `/v3/projects/2e11b6eb820642d48a22afd352d8a9dd` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:09.737283 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3cf1ef-8c23-450a-8922-49127d78c2c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e11b6eb820642d48a22afd352d8a9dd)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:09.739209 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3cf1ef-8c23-450a-8922-49127d78c2c2 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:09.744167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f67e91-fc1e-4805-bd91-591fc3fc6048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:09.745599 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f67e91-fc1e-4805-bd91-591fc3fc6048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:09.746524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f67e91-fc1e-4805-bd91-591fc3fc6048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:09.748064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f67e91-fc1e-4805-bd91-591fc3fc6048 admin admin] PATH_INFO: `/v3/projects/5d7a045b3bfa4af18bf51c8d525f2758` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:09.750298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f67e91-fc1e-4805-bd91-591fc3fc6048 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d7a045b3bfa4af18bf51c8d525f2758)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:09.753462 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f67e91-fc1e-4805-bd91-591fc3fc6048 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:09.802008 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-8f3cf1ef-8c23-450a-8922-49127d78c2c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e11b6eb820642d48a22afd352d8a9dd', 'request_id': 'req-8f3cf1ef-8c23-450a-8922-49127d78c2c2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:09.814020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-8f3cf1ef-8c23-450a-8922-49127d78c2c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e11b6eb820642d48a22afd352d8a9dd', 'request_id': 'req-8f3cf1ef-8c23-450a-8922-49127d78c2c2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:09.817443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d5f67e91-fc1e-4805-bd91-591fc3fc6048 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d7a045b3bfa4af18bf51c8d525f2758', 'request_id': 'req-d5f67e91-fc1e-4805-bd91-591fc3fc6048'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:09.826524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-8f3cf1ef-8c23-450a-8922-49127d78c2c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e11b6eb820642d48a22afd352d8a9dd', 'request_id': 'req-8f3cf1ef-8c23-450a-8922-49127d78c2c2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:09.830669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d5f67e91-fc1e-4805-bd91-591fc3fc6048 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d7a045b3bfa4af18bf51c8d525f2758', 'request_id': 'req-d5f67e91-fc1e-4805-bd91-591fc3fc6048'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:09.836557 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d5f67e91-fc1e-4805-bd91-591fc3fc6048 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d7a045b3bfa4af18bf51c8d525f2758', 'request_id': 'req-d5f67e91-fc1e-4805-bd91-591fc3fc6048'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:09.842133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-8f3cf1ef-8c23-450a-8922-49127d78c2c2 admin admin] The token cache is being invalidate because project 2e11b6eb820642d48a22afd352d8a9dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:40:09.842425 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-8f3cf1ef-8c23-450a-8922-49127d78c2c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:09.843075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-8f3cf1ef-8c23-450a-8922-49127d78c2c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:09.845367 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1103/2211] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:40:09 2022] DELETE /identity/v3/projects/2e11b6eb820642d48a22afd352d8a9dd => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:09.850227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb38918-a8b7-40db-8e10-36e19d288ae2 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:09.850629 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb38918-a8b7-40db-8e10-36e19d288ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:09.850909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb38918-a8b7-40db-8e10-36e19d288ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:09.851713 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d5f67e91-fc1e-4805-bd91-591fc3fc6048 admin admin] The token cache is being invalidate because project 5d7a045b3bfa4af18bf51c8d525f2758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:40:09.852063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d5f67e91-fc1e-4805-bd91-591fc3fc6048 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:09.852819 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d5f67e91-fc1e-4805-bd91-591fc3fc6048 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:09.855281 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1109/2212] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:40:09 2022] DELETE /identity/v3/projects/5d7a045b3bfa4af18bf51c8d525f2758 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:40:09.859701 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194e038-5711-4a6d-a514-3a7615c143d6 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:09.860040 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194e038-5711-4a6d-a514-3a7615c143d6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:09.860316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194e038-5711-4a6d-a514-3a7615c143d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:09.871366 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-9eb38918-a8b7-40db-8e10-36e19d288ae2 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:09.873406 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-9eb38918-a8b7-40db-8e10-36e19d288ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:09.882333 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-0194e038-5711-4a6d-a514-3a7615c143d6 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:09.883492 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0194e038-5711-4a6d-a514-3a7615c143d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:09.974899 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1110/2213] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:09.976577 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1104/2214] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:40:09.984014 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6389f-8786-48e6-82bc-12ec9f424627 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:09.984014 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6389f-8786-48e6-82bc-12ec9f424627 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:09.984014 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6389f-8786-48e6-82bc-12ec9f424627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:09.986642 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cf732-a2d2-43d5-bab6-3f3e5b7b0af4 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:09.987103 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cf732-a2d2-43d5-bab6-3f3e5b7b0af4 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:09.987534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cf732-a2d2-43d5-bab6-3f3e5b7b0af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.003698 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-aca6389f-8786-48e6-82bc-12ec9f424627 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:10.005042 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-aca6389f-8786-48e6-82bc-12ec9f424627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:10.011324 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-cc6cf732-a2d2-43d5-bab6-3f3e5b7b0af4 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:10.013096 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-cc6cf732-a2d2-43d5-bab6-3f3e5b7b0af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:10.015140 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1111/2215] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:10.019822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6389f-8786-48e6-82bc-12ec9f424627 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:10.029420 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1105/2216] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:10.030447 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cf732-a2d2-43d5-bab6-3f3e5b7b0af4 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:10.033943 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6389f-8786-48e6-82bc-12ec9f424627 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:10.039171 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78858ab4-52d4-4c24-b863-3c2704f98efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:10.039621 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78858ab4-52d4-4c24-b863-3c2704f98efc admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.040109 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78858ab4-52d4-4c24-b863-3c2704f98efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.041117 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cf732-a2d2-43d5-bab6-3f3e5b7b0af4 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:10.041237 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78858ab4-52d4-4c24-b863-3c2704f98efc admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.042813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78858ab4-52d4-4c24-b863-3c2704f98efc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:40:10.042813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78858ab4-52d4-4c24-b863-3c2704f98efc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:10.044480 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78858ab4-52d4-4c24-b863-3c2704f98efc admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:10.046187 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:10.047585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbe333-8f4b-4017-9e46-c9b5430921ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:10.048488 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbe333-8f4b-4017-9e46-c9b5430921ea admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.049046 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbe333-8f4b-4017-9e46-c9b5430921ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.049622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbe333-8f4b-4017-9e46-c9b5430921ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.050291 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dbe333-8f4b-4017-9e46-c9b5430921ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:40:10.050672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dbe333-8f4b-4017-9e46-c9b5430921ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:10.052357 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dbe333-8f4b-4017-9e46-c9b5430921ea admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:10.060813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1112/2217] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:40:10.065422 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78858ab4-52d4-4c24-b863-3c2704f98efc admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:10.072639 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1106/2218] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:40:10.073907 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbe333-8f4b-4017-9e46-c9b5430921ea admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:10.082705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78858ab4-52d4-4c24-b863-3c2704f98efc admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:10.088120 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbe333-8f4b-4017-9e46-c9b5430921ea admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:10.091524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8cea3a-66b6-4569-94b6-f56ea589dd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:10.092488 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8cea3a-66b6-4569-94b6-f56ea589dd8a admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.092907 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8cea3a-66b6-4569-94b6-f56ea589dd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.093529 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8cea3a-66b6-4569-94b6-f56ea589dd8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.094224 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8cea3a-66b6-4569-94b6-f56ea589dd8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:10.095803 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff0977c-2ef7-4605-ba9a-7d0ef0cde79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:10.096456 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8cea3a-66b6-4569-94b6-f56ea589dd8a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:10.096753 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0977c-2ef7-4605-ba9a-7d0ef0cde79c admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.097131 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0977c-2ef7-4605-ba9a-7d0ef0cde79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.098028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0977c-2ef7-4605-ba9a-7d0ef0cde79c admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.098455 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff0977c-2ef7-4605-ba9a-7d0ef0cde79c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:10.101608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff0977c-2ef7-4605-ba9a-7d0ef0cde79c admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:10.152513 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1113/2219] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:40:10 2022] POST /identity/v3/projects => generated 401 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:40:10.153063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1107/2220] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:40:10 2022] POST /identity/v3/projects => generated 381 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:40:10.156239 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8cea3a-66b6-4569-94b6-f56ea589dd8a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:10.156764 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff0977c-2ef7-4605-ba9a-7d0ef0cde79c admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:10.175039 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:10.242761 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff0977c-2ef7-4605-ba9a-7d0ef0cde79c admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:10.243813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8cea3a-66b6-4569-94b6-f56ea589dd8a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:10.248931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f942c3-2ec3-4a30-90d1-6b603f5c260c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:10.249615 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f942c3-2ec3-4a30-90d1-6b603f5c260c admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.249905 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f942c3-2ec3-4a30-90d1-6b603f5c260c admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.250184 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f942c3-2ec3-4a30-90d1-6b603f5c260c admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.250880 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f942c3-2ec3-4a30-90d1-6b603f5c260c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:10.251982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf71451-837b-4c50-a42b-064a93d920a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:10.252872 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f942c3-2ec3-4a30-90d1-6b603f5c260c admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:10.253020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf71451-837b-4c50-a42b-064a93d920a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.253233 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf71451-837b-4c50-a42b-064a93d920a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.253517 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf71451-837b-4c50-a42b-064a93d920a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.254349 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf71451-837b-4c50-a42b-064a93d920a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:10.256871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf71451-837b-4c50-a42b-064a93d920a4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:10.280479 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1108/2221] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:40:10 2022] POST /identity/v3/users => generated 291 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:40:10.284972 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f942c3-2ec3-4a30-90d1-6b603f5c260c admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:10.298367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1114/2222] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:40:10 2022] POST /identity/v3/users => generated 301 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:40:10.298710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f942c3-2ec3-4a30-90d1-6b603f5c260c admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:10.302992 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf71451-837b-4c50-a42b-064a93d920a4 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:10.307354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de250f91-6522-4a9e-83e6-6718a2088d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:10.310176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de250f91-6522-4a9e-83e6-6718a2088d80 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.310176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de250f91-6522-4a9e-83e6-6718a2088d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.310176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de250f91-6522-4a9e-83e6-6718a2088d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.310176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de250f91-6522-4a9e-83e6-6718a2088d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:10.311538 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de250f91-6522-4a9e-83e6-6718a2088d80 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:10.319209 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf71451-837b-4c50-a42b-064a93d920a4 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:10.325013 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c5e26-6e06-4df5-9212-f5ada1e5cdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:10.326163 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c5e26-6e06-4df5-9212-f5ada1e5cdf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.326304 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1109/2223] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:40:10.326644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c5e26-6e06-4df5-9212-f5ada1e5cdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.327014 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c5e26-6e06-4df5-9212-f5ada1e5cdf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.327544 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e957ad-4da1-493c-aa0a-b5fbe59e4518 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.327862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e957ad-4da1-493c-aa0a-b5fbe59e4518 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.328173 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c5e26-6e06-4df5-9212-f5ada1e5cdf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:10.328288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e957ad-4da1-493c-aa0a-b5fbe59e4518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.330005 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c5e26-6e06-4df5-9212-f5ada1e5cdf0 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:10.341954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1115/2224] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:40:10.342921 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c5e26-6e06-4df5-9212-f5ada1e5cdf0 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:10.347644 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-22e957ad-4da1-493c-aa0a-b5fbe59e4518 None None] MFA Rules not processed for user `f269e146c844480c822347c3a38d103b`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:10.348893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-22e957ad-4da1-493c-aa0a-b5fbe59e4518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:10.353538 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c5e26-6e06-4df5-9212-f5ada1e5cdf0 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:10.358538 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75f805-00c3-4f4e-b8a2-64ed046bcdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:10.359307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75f805-00c3-4f4e-b8a2-64ed046bcdc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.359658 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75f805-00c3-4f4e-b8a2-64ed046bcdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.359983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75f805-00c3-4f4e-b8a2-64ed046bcdc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.360717 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75f805-00c3-4f4e-b8a2-64ed046bcdc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:10.362317 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75f805-00c3-4f4e-b8a2-64ed046bcdc3 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:10.375734 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1116/2225] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:40:10.376701 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75f805-00c3-4f4e-b8a2-64ed046bcdc3 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:10.391173 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75f805-00c3-4f4e-b8a2-64ed046bcdc3 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:10.400112 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c7e02-996d-4482-b3ee-2137b625bf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:10.400802 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c7e02-996d-4482-b3ee-2137b625bf00 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.401194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c7e02-996d-4482-b3ee-2137b625bf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.401473 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c7e02-996d-4482-b3ee-2137b625bf00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.402219 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c7e02-996d-4482-b3ee-2137b625bf00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:10.403960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c7e02-996d-4482-b3ee-2137b625bf00 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:10.417476 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1117/2226] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:40:10.420259 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c7e02-996d-4482-b3ee-2137b625bf00 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:10.431913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c7e02-996d-4482-b3ee-2137b625bf00 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:10.433714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1110/2227] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:10.434355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e957ad-4da1-493c-aa0a-b5fbe59e4518 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:10.444247 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8154f-fa6e-4326-8b73-47cf32a911a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:10.444891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8154f-fa6e-4326-8b73-47cf32a911a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.445175 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8154f-fa6e-4326-8b73-47cf32a911a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.445523 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8154f-fa6e-4326-8b73-47cf32a911a8 admin admin] PATH_INFO: `/v3/projects/e290529eb1b848f79e013701f1ab7a65/users/7b0809f17d2345dd9042a74123fac148/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.449298 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e957ad-4da1-493c-aa0a-b5fbe59e4518 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:10.455946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07edff9-29c6-4c66-a657-ef224011c17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:10.457340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07edff9-29c6-4c66-a657-ef224011c17d admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.457633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07edff9-29c6-4c66-a657-ef224011c17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.457908 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07edff9-29c6-4c66-a657-ef224011c17d admin admin] PATH_INFO: `/v3/projects/3d90957df61d421297f92954f9bd3ed4/users/2b3636afff90484caea7d1bcc0f1e801/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.470051 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8154f-fa6e-4326-8b73-47cf32a911a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e290529eb1b848f79e013701f1ab7a65, user_id=7b0809f17d2345dd9042a74123fac148, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:10.473871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8154f-fa6e-4326-8b73-47cf32a911a8 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:10.477261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07edff9-29c6-4c66-a657-ef224011c17d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d90957df61d421297f92954f9bd3ed4, user_id=2b3636afff90484caea7d1bcc0f1e801, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:10.482694 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07edff9-29c6-4c66-a657-ef224011c17d admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:10.488465 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1118/2228] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:40:10 2022] PUT /identity/v3/projects/e290529eb1b848f79e013701f1ab7a65/users/7b0809f17d2345dd9042a74123fac148/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:10.496283 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1111/2229] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:40:10 2022] PUT /identity/v3/projects/3d90957df61d421297f92954f9bd3ed4/users/2b3636afff90484caea7d1bcc0f1e801/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:10.499031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf3d48-5e60-470f-90ec-1249e84a3969 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.499324 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf3d48-5e60-470f-90ec-1249e84a3969 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.499679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf3d48-5e60-470f-90ec-1249e84a3969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.510948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a96fe2-acc4-4392-a700-8cf831e09442 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.511480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a96fe2-acc4-4392-a700-8cf831e09442 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.511879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a96fe2-acc4-4392-a700-8cf831e09442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.537560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-eddf3d48-5e60-470f-90ec-1249e84a3969 None None] MFA Rules not processed for user `7b0809f17d2345dd9042a74123fac148`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:10.538783 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-eddf3d48-5e60-470f-90ec-1249e84a3969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:10.547161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-e6a96fe2-acc4-4392-a700-8cf831e09442 None None] MFA Rules not processed for user `2b3636afff90484caea7d1bcc0f1e801`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:10.548405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-e6a96fe2-acc4-4392-a700-8cf831e09442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:10.599960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:10.612572 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:10.613133 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1119/2230] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:10.617915 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf3d48-5e60-470f-90ec-1249e84a3969 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:10.628096 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1112/2231] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:10.630054 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a96fe2-acc4-4392-a700-8cf831e09442 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:10.688717 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf3d48-5e60-470f-90ec-1249e84a3969 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:10.693932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd0821-db99-493e-8bd9-3c8aa0a3b33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:10.694595 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd0821-db99-493e-8bd9-3c8aa0a3b33d admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.694868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd0821-db99-493e-8bd9-3c8aa0a3b33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.695380 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd0821-db99-493e-8bd9-3c8aa0a3b33d admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.695764 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a96fe2-acc4-4392-a700-8cf831e09442 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:10.696161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd0821-db99-493e-8bd9-3c8aa0a3b33d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:10.698174 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd0821-db99-493e-8bd9-3c8aa0a3b33d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:10.700572 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529957de-6371-4292-bce1-8da08e677c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:10.701236 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529957de-6371-4292-bce1-8da08e677c94 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.701519 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529957de-6371-4292-bce1-8da08e677c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.701798 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529957de-6371-4292-bce1-8da08e677c94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.702483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529957de-6371-4292-bce1-8da08e677c94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:10.704455 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529957de-6371-4292-bce1-8da08e677c94 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:10.752090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1120/2232] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:40:10 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:40:10.753033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd0821-db99-493e-8bd9-3c8aa0a3b33d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:10.761572 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1113/2233] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:40:10 2022] POST /identity/v3/projects => generated 399 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:40:10.762544 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529957de-6371-4292-bce1-8da08e677c94 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:10.866715 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd0821-db99-493e-8bd9-3c8aa0a3b33d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:10.871861 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1945b-c725-46fa-a5b3-0ac06de29c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:10.872620 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1945b-c725-46fa-a5b3-0ac06de29c2e admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.872998 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1945b-c725-46fa-a5b3-0ac06de29c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.873406 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1945b-c725-46fa-a5b3-0ac06de29c2e admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.874117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1945b-c725-46fa-a5b3-0ac06de29c2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:10.876264 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1945b-c725-46fa-a5b3-0ac06de29c2e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:10.878609 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529957de-6371-4292-bce1-8da08e677c94 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:10.883267 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b25dd9-e651-4cbf-aeba-6c9da9570cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:10.883989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b25dd9-e651-4cbf-aeba-6c9da9570cf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.884271 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b25dd9-e651-4cbf-aeba-6c9da9570cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.884546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b25dd9-e651-4cbf-aeba-6c9da9570cf3 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.885228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b25dd9-e651-4cbf-aeba-6c9da9570cf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:10.887110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b25dd9-e651-4cbf-aeba-6c9da9570cf3 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:10.897860 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1121/2234] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:40:10 2022] POST /identity/v3/users => generated 297 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:40:10.903275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1945b-c725-46fa-a5b3-0ac06de29c2e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:10.911236 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1114/2235] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:40:10 2022] POST /identity/v3/users => generated 306 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:40:10.912280 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b25dd9-e651-4cbf-aeba-6c9da9570cf3 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:10.913971 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1945b-c725-46fa-a5b3-0ac06de29c2e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:10.920117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec533f7-a64f-4ffb-99d5-fbe2cf39e044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:10.920779 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec533f7-a64f-4ffb-99d5-fbe2cf39e044 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.921061 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec533f7-a64f-4ffb-99d5-fbe2cf39e044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.921331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec533f7-a64f-4ffb-99d5-fbe2cf39e044 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.922052 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec533f7-a64f-4ffb-99d5-fbe2cf39e044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:10.923104 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b25dd9-e651-4cbf-aeba-6c9da9570cf3 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:10.923647 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec533f7-a64f-4ffb-99d5-fbe2cf39e044 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:10.925120 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:10.928328 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960878f-fce0-45b4-8f36-4dfc2ee3699d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:10.929104 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960878f-fce0-45b4-8f36-4dfc2ee3699d admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.929374 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960878f-fce0-45b4-8f36-4dfc2ee3699d admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.930046 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960878f-fce0-45b4-8f36-4dfc2ee3699d admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.930361 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960878f-fce0-45b4-8f36-4dfc2ee3699d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:10.933010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960878f-fce0-45b4-8f36-4dfc2ee3699d admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:10.935274 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:10.935982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1122/2236] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:40:10.940060 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec533f7-a64f-4ffb-99d5-fbe2cf39e044 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:10.949536 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1115/2237] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:40:10.952413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec533f7-a64f-4ffb-99d5-fbe2cf39e044 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:10.953383 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960878f-fce0-45b4-8f36-4dfc2ee3699d admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:10.957416 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c7dd4-70f4-44d3-8249-bee61811de19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:10.958703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c7dd4-70f4-44d3-8249-bee61811de19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.959911 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c7dd4-70f4-44d3-8249-bee61811de19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.960347 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c7dd4-70f4-44d3-8249-bee61811de19 admin admin] PATH_INFO: `/v3/projects/0409bd67a2f941af95784c3aafd2b5fc/users/a8c3f80302e344c091643fa39c01c99c/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.963755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960878f-fce0-45b4-8f36-4dfc2ee3699d admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:10.968357 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e2087-f1bb-4172-8708-e006f0dfe402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:10.969321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e2087-f1bb-4172-8708-e006f0dfe402 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:10.969608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e2087-f1bb-4172-8708-e006f0dfe402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:10.969885 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e2087-f1bb-4172-8708-e006f0dfe402 admin admin] PATH_INFO: `/v3/projects/9b75596c509b485181d77aed43ac114d/users/2211508a748c4dcd828bdcfb19a1214f/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:10.979368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c7dd4-70f4-44d3-8249-bee61811de19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0409bd67a2f941af95784c3aafd2b5fc, user_id=a8c3f80302e344c091643fa39c01c99c, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:10.983403 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c7dd4-70f4-44d3-8249-bee61811de19 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:10.998848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e2087-f1bb-4172-8708-e006f0dfe402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b75596c509b485181d77aed43ac114d, user_id=2211508a748c4dcd828bdcfb19a1214f, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:11.002091 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e2087-f1bb-4172-8708-e006f0dfe402 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:11.003261 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1123/2238] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:40:10 2022] PUT /identity/v3/projects/0409bd67a2f941af95784c3aafd2b5fc/users/a8c3f80302e344c091643fa39c01c99c/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:40:11.006999 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c7dd4-70f4-44d3-8249-bee61811de19 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:11.026330 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1116/2239] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:40:10 2022] PUT /identity/v3/projects/9b75596c509b485181d77aed43ac114d/users/2211508a748c4dcd828bdcfb19a1214f/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:40:11.030242 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e2087-f1bb-4172-8708-e006f0dfe402 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:11.089041 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c7dd4-70f4-44d3-8249-bee61811de19 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:11.102262 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e2087-f1bb-4172-8708-e006f0dfe402 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:11.107931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2cfc17-fc00-4fd5-98ee-003c791779ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:11.108913 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2cfc17-fc00-4fd5-98ee-003c791779ff admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:11.109436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2cfc17-fc00-4fd5-98ee-003c791779ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:11.109945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2cfc17-fc00-4fd5-98ee-003c791779ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:11.110961 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2cfc17-fc00-4fd5-98ee-003c791779ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:11.112609 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2cfc17-fc00-4fd5-98ee-003c791779ff admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:11.130531 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1117/2240] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:40:11.131263 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2cfc17-fc00-4fd5-98ee-003c791779ff admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:11.141907 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2cfc17-fc00-4fd5-98ee-003c791779ff admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:11.147188 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08554862-4742-40ba-b91a-5602d0a8204b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:11.148103 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08554862-4742-40ba-b91a-5602d0a8204b admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:11.148384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08554862-4742-40ba-b91a-5602d0a8204b admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:11.148657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08554862-4742-40ba-b91a-5602d0a8204b admin admin] PATH_INFO: `/v3/projects/9b75596c509b485181d77aed43ac114d/users/2211508a748c4dcd828bdcfb19a1214f/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:11.150963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08554862-4742-40ba-b91a-5602d0a8204b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b75596c509b485181d77aed43ac114d, user_id=2211508a748c4dcd828bdcfb19a1214f, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:11.153607 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08554862-4742-40ba-b91a-5602d0a8204b admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:11.157737 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353592b-3f3f-4dcf-b125-265cc851a427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:11.158521 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353592b-3f3f-4dcf-b125-265cc851a427 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:11.158883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353592b-3f3f-4dcf-b125-265cc851a427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:11.159222 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353592b-3f3f-4dcf-b125-265cc851a427 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:11.160092 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d353592b-3f3f-4dcf-b125-265cc851a427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:11.161756 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d353592b-3f3f-4dcf-b125-265cc851a427 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:11.166767 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1118/2241] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:40:11 2022] PUT /identity/v3/projects/9b75596c509b485181d77aed43ac114d/users/2211508a748c4dcd828bdcfb19a1214f/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:11.174192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1124/2242] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:40:11.177649 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea73cce-c7b4-4984-98a5-63fa84d79ad7 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:11.178011 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea73cce-c7b4-4984-98a5-63fa84d79ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:11.178209 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353592b-3f3f-4dcf-b125-265cc851a427 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:11.178792 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea73cce-c7b4-4984-98a5-63fa84d79ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:11.189311 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353592b-3f3f-4dcf-b125-265cc851a427 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:11.214526 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-7ea73cce-c7b4-4984-98a5-63fa84d79ad7 None None] MFA Rules not processed for user `2211508a748c4dcd828bdcfb19a1214f`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:11.215763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-7ea73cce-c7b4-4984-98a5-63fa84d79ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:11.270331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767986f9-2708-4a75-ab48-6962fb45d67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:11.271241 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767986f9-2708-4a75-ab48-6962fb45d67e admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:11.271605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767986f9-2708-4a75-ab48-6962fb45d67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:11.271973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767986f9-2708-4a75-ab48-6962fb45d67e admin admin] PATH_INFO: `/v3/projects/0409bd67a2f941af95784c3aafd2b5fc/users/a8c3f80302e344c091643fa39c01c99c/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:11.274181 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767986f9-2708-4a75-ab48-6962fb45d67e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0409bd67a2f941af95784c3aafd2b5fc, user_id=a8c3f80302e344c091643fa39c01c99c, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:11.277400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767986f9-2708-4a75-ab48-6962fb45d67e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:11.288749 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1125/2243] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:40:11 2022] PUT /identity/v3/projects/0409bd67a2f941af95784c3aafd2b5fc/users/a8c3f80302e344c091643fa39c01c99c/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:40:11.298040 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1119/2244] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:11 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:11.300391 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e1626-50e7-4ddd-a092-1b7405877f68 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:11.300684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e1626-50e7-4ddd-a092-1b7405877f68 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:11.300960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e1626-50e7-4ddd-a092-1b7405877f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:11.309633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea73cce-c7b4-4984-98a5-63fa84d79ad7 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:11.311019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-7ea73cce-c7b4-4984-98a5-63fa84d79ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:11.321787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:11.332215 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-583e1626-50e7-4ddd-a092-1b7405877f68 None None] MFA Rules not processed for user `a8c3f80302e344c091643fa39c01c99c`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:11.333442 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-583e1626-50e7-4ddd-a092-1b7405877f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:11.359628 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:11.396834 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea73cce-c7b4-4984-98a5-63fa84d79ad7 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:11.410193 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1126/2245] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:11.425525 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e1626-50e7-4ddd-a092-1b7405877f68 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:11.427013 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-583e1626-50e7-4ddd-a092-1b7405877f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:11.455980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38483a46-23b8-41d5-bb49-07d45622399b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:11.456630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38483a46-23b8-41d5-bb49-07d45622399b service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:11.456911 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38483a46-23b8-41d5-bb49-07d45622399b service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:11.457215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38483a46-23b8-41d5-bb49-07d45622399b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:11.494083 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e1626-50e7-4ddd-a092-1b7405877f68 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:11.499272 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefabe2-f7fc-4225-b219-42b965553201 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:11.500043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefabe2-f7fc-4225-b219-42b965553201 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:11.500331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefabe2-f7fc-4225-b219-42b965553201 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:11.500841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefabe2-f7fc-4225-b219-42b965553201 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:11.511272 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faefabe2-f7fc-4225-b219-42b965553201 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:11.512742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faefabe2-f7fc-4225-b219-42b965553201 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:11.523202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1127/2246] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:40:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:40:11.533185 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38483a46-23b8-41d5-bb49-07d45622399b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:11.534617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38483a46-23b8-41d5-bb49-07d45622399b service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:11.544955 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1120/2247] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:40:11 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:40:11.676743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefabe2-f7fc-4225-b219-42b965553201 service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:11.691656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefabe2-f7fc-4225-b219-42b965553201 service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:11.697700 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179dac2-9934-4705-86a2-49b3f0dbd31a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:11.698359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179dac2-9934-4705-86a2-49b3f0dbd31a service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:11.698734 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179dac2-9934-4705-86a2-49b3f0dbd31a service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:11.699236 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179dac2-9934-4705-86a2-49b3f0dbd31a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:11.768117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0179dac2-9934-4705-86a2-49b3f0dbd31a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:11.769815 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0179dac2-9934-4705-86a2-49b3f0dbd31a service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:11.780148 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1128/2248] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:40:11 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:40:12.334910 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38483a46-23b8-41d5-bb49-07d45622399b service cinder] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:12.420093 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38483a46-23b8-41d5-bb49-07d45622399b service cinder] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:12.426165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3725f-edd8-47bc-934e-28f967a480c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:12.427049 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3725f-edd8-47bc-934e-28f967a480c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:12.427337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3725f-edd8-47bc-934e-28f967a480c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:12.427613 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3725f-edd8-47bc-934e-28f967a480c5 admin admin] PATH_INFO: `/v3/users/7b0809f17d2345dd9042a74123fac148` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:12.429166 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3725f-edd8-47bc-934e-28f967a480c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b0809f17d2345dd9042a74123fac148)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:12.431710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3725f-edd8-47bc-934e-28f967a480c5 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:12.486698 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:12.498240 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-51f3725f-edd8-47bc-934e-28f967a480c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b0809f17d2345dd9042a74123fac148'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:12.506345 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-51f3725f-edd8-47bc-934e-28f967a480c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b0809f17d2345dd9042a74123fac148'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:12.516865 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-51f3725f-edd8-47bc-934e-28f967a480c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b0809f17d2345dd9042a74123fac148'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:12.517439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-51f3725f-edd8-47bc-934e-28f967a480c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b0809f17d2345dd9042a74123fac148'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:12.532202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-51f3725f-edd8-47bc-934e-28f967a480c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b0809f17d2345dd9042a74123fac148'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:12.538751 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1121/2249] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:40:12 2022] DELETE /identity/v3/users/7b0809f17d2345dd9042a74123fac148 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:40:12.544648 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecb145-f906-42fb-a787-50e8f88dbd8b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:12.545040 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecb145-f906-42fb-a787-50e8f88dbd8b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:12.545388 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecb145-f906-42fb-a787-50e8f88dbd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:12.572710 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-41ecb145-f906-42fb-a787-50e8f88dbd8b None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:12.574242 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-41ecb145-f906-42fb-a787-50e8f88dbd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:12.600946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:12.646559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1129/2250] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:12.656842 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3725f-edd8-47bc-934e-28f967a480c5 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:12.658226 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-51f3725f-edd8-47bc-934e-28f967a480c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:12.757278 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3725f-edd8-47bc-934e-28f967a480c5 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:12.761784 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb1800-2eab-4ca0-9d51-92006e5d3c42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:12.762530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb1800-2eab-4ca0-9d51-92006e5d3c42 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:12.762883 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb1800-2eab-4ca0-9d51-92006e5d3c42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:12.763227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb1800-2eab-4ca0-9d51-92006e5d3c42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:12.772795 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb1800-2eab-4ca0-9d51-92006e5d3c42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:12.774430 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb1800-2eab-4ca0-9d51-92006e5d3c42 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:12.784474 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1122/2251] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:40:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:40:13.107593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ecb145-f906-42fb-a787-50e8f88dbd8b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:13.109064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-41ecb145-f906-42fb-a787-50e8f88dbd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:13.121517 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ecb145-f906-42fb-a787-50e8f88dbd8b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:13.129755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c821077-09ea-43dd-8a59-6e0d250079f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:13.130929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c821077-09ea-43dd-8a59-6e0d250079f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:13.131139 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c821077-09ea-43dd-8a59-6e0d250079f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:13.131414 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c821077-09ea-43dd-8a59-6e0d250079f3 admin admin] PATH_INFO: `/v3/projects/e290529eb1b848f79e013701f1ab7a65` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:13.132760 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c821077-09ea-43dd-8a59-6e0d250079f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e290529eb1b848f79e013701f1ab7a65)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:13.134923 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c821077-09ea-43dd-8a59-6e0d250079f3 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:13.188498 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4c821077-09ea-43dd-8a59-6e0d250079f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e290529eb1b848f79e013701f1ab7a65', 'request_id': 'req-4c821077-09ea-43dd-8a59-6e0d250079f3'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:13.198851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4c821077-09ea-43dd-8a59-6e0d250079f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e290529eb1b848f79e013701f1ab7a65', 'request_id': 'req-4c821077-09ea-43dd-8a59-6e0d250079f3'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:13.203565 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4c821077-09ea-43dd-8a59-6e0d250079f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e290529eb1b848f79e013701f1ab7a65', 'request_id': 'req-4c821077-09ea-43dd-8a59-6e0d250079f3'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:13.217329 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4c821077-09ea-43dd-8a59-6e0d250079f3 admin admin] The token cache is being invalidate because project e290529eb1b848f79e013701f1ab7a65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:40:13.217615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4c821077-09ea-43dd-8a59-6e0d250079f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:13.218275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4c821077-09ea-43dd-8a59-6e0d250079f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:13.220734 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1130/2252] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:40:13 2022] DELETE /identity/v3/projects/e290529eb1b848f79e013701f1ab7a65 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:13.224740 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb1800-2eab-4ca0-9d51-92006e5d3c42 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:13.234981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb1800-2eab-4ca0-9d51-92006e5d3c42 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:13.294672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca7799-d2b6-4f1e-a7a0-80d259c9cff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:13.295550 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca7799-d2b6-4f1e-a7a0-80d259c9cff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:13.295882 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca7799-d2b6-4f1e-a7a0-80d259c9cff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:13.296167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca7799-d2b6-4f1e-a7a0-80d259c9cff5 admin admin] PATH_INFO: `/v3/users/a8c3f80302e344c091643fa39c01c99c` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:13.297362 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca7799-d2b6-4f1e-a7a0-80d259c9cff5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8c3f80302e344c091643fa39c01c99c)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:13.299279 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca7799-d2b6-4f1e-a7a0-80d259c9cff5 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:13.339431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:13.350368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e0ca7799-d2b6-4f1e-a7a0-80d259c9cff5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8c3f80302e344c091643fa39c01c99c'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:13.358499 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e0ca7799-d2b6-4f1e-a7a0-80d259c9cff5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8c3f80302e344c091643fa39c01c99c'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:13.367270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e0ca7799-d2b6-4f1e-a7a0-80d259c9cff5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8c3f80302e344c091643fa39c01c99c'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:13.368082 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e0ca7799-d2b6-4f1e-a7a0-80d259c9cff5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8c3f80302e344c091643fa39c01c99c'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:13.401002 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e0ca7799-d2b6-4f1e-a7a0-80d259c9cff5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8c3f80302e344c091643fa39c01c99c'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:13.407810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1123/2253] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:40:13 2022] DELETE /identity/v3/users/a8c3f80302e344c091643fa39c01c99c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:13.848440 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c821077-09ea-43dd-8a59-6e0d250079f3 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:13.850201 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4c821077-09ea-43dd-8a59-6e0d250079f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:13.867557 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c821077-09ea-43dd-8a59-6e0d250079f3 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:13.942273 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f474ccd-8589-4f0a-bfeb-a3aecd5f2459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:13.943381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f474ccd-8589-4f0a-bfeb-a3aecd5f2459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:13.944269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f474ccd-8589-4f0a-bfeb-a3aecd5f2459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:13.944269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f474ccd-8589-4f0a-bfeb-a3aecd5f2459 admin admin] PATH_INFO: `/v3/projects/0409bd67a2f941af95784c3aafd2b5fc` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:13.945353 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f474ccd-8589-4f0a-bfeb-a3aecd5f2459 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0409bd67a2f941af95784c3aafd2b5fc)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:13.947346 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f474ccd-8589-4f0a-bfeb-a3aecd5f2459 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:14.049276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7f474ccd-8589-4f0a-bfeb-a3aecd5f2459 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0409bd67a2f941af95784c3aafd2b5fc', 'request_id': 'req-7f474ccd-8589-4f0a-bfeb-a3aecd5f2459'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:14.054063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:14.059634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7f474ccd-8589-4f0a-bfeb-a3aecd5f2459 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0409bd67a2f941af95784c3aafd2b5fc', 'request_id': 'req-7f474ccd-8589-4f0a-bfeb-a3aecd5f2459'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:14.077817 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7f474ccd-8589-4f0a-bfeb-a3aecd5f2459 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0409bd67a2f941af95784c3aafd2b5fc', 'request_id': 'req-7f474ccd-8589-4f0a-bfeb-a3aecd5f2459'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:14.080931 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7f474ccd-8589-4f0a-bfeb-a3aecd5f2459 admin admin] The token cache is being invalidate because project 0409bd67a2f941af95784c3aafd2b5fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:40:14.081350 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7f474ccd-8589-4f0a-bfeb-a3aecd5f2459 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:14.082024 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7f474ccd-8589-4f0a-bfeb-a3aecd5f2459 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:14.083613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1131/2254] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:40:13 2022] DELETE /identity/v3/projects/0409bd67a2f941af95784c3aafd2b5fc => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:14.089400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc84b9-0240-4adb-9c59-8c0ba7d609de None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:14.089770 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc84b9-0240-4adb-9c59-8c0ba7d609de None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:14.090120 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc84b9-0240-4adb-9c59-8c0ba7d609de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:14.133104 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-d6cc84b9-0240-4adb-9c59-8c0ba7d609de None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:14.134845 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-d6cc84b9-0240-4adb-9c59-8c0ba7d609de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:14.198551 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1124/2255] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:14.206577 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c9a04-711b-4627-8278-40c2e6860dd6 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:14.207105 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c9a04-711b-4627-8278-40c2e6860dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:14.207374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c9a04-711b-4627-8278-40c2e6860dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:14.226227 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2f3c9a04-711b-4627-8278-40c2e6860dd6 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:14.228077 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2f3c9a04-711b-4627-8278-40c2e6860dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:14.236547 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1132/2256] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:14.240746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc84b9-0240-4adb-9c59-8c0ba7d609de None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:14.250996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc84b9-0240-4adb-9c59-8c0ba7d609de None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:14.255511 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714752c-08a8-4010-9fb8-7592ffd2ef54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:14.256193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714752c-08a8-4010-9fb8-7592ffd2ef54 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:14.256478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714752c-08a8-4010-9fb8-7592ffd2ef54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:14.256756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714752c-08a8-4010-9fb8-7592ffd2ef54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:14.257506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5714752c-08a8-4010-9fb8-7592ffd2ef54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:40:14.257809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5714752c-08a8-4010-9fb8-7592ffd2ef54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:14.259599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5714752c-08a8-4010-9fb8-7592ffd2ef54 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:14.278193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1125/2257] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:40:14.282355 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c9a04-711b-4627-8278-40c2e6860dd6 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:14.294664 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c9a04-711b-4627-8278-40c2e6860dd6 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:14.299309 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30016d97-213b-49ee-bde9-81688c68190d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:14.299996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30016d97-213b-49ee-bde9-81688c68190d admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:14.300280 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30016d97-213b-49ee-bde9-81688c68190d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:14.300558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30016d97-213b-49ee-bde9-81688c68190d admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:14.301269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30016d97-213b-49ee-bde9-81688c68190d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:14.303136 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30016d97-213b-49ee-bde9-81688c68190d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:14.330802 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1133/2258] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:40:14 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:40:14.333192 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714752c-08a8-4010-9fb8-7592ffd2ef54 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:14.343804 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714752c-08a8-4010-9fb8-7592ffd2ef54 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:14.401517 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289ebdf-9445-4519-8604-4d48e56712c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:14.402192 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289ebdf-9445-4519-8604-4d48e56712c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:14.402481 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289ebdf-9445-4519-8604-4d48e56712c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:14.402787 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289ebdf-9445-4519-8604-4d48e56712c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:14.403488 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289ebdf-9445-4519-8604-4d48e56712c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:14.405588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289ebdf-9445-4519-8604-4d48e56712c4 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:14.425074 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1126/2259] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:40:14 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:40:14.430150 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30016d97-213b-49ee-bde9-81688c68190d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:14.442544 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30016d97-213b-49ee-bde9-81688c68190d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:14.448137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756991e5-ae14-4bfd-b683-d5411d989559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:14.448796 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756991e5-ae14-4bfd-b683-d5411d989559 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:14.449079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756991e5-ae14-4bfd-b683-d5411d989559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:14.449352 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756991e5-ae14-4bfd-b683-d5411d989559 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:14.450071 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756991e5-ae14-4bfd-b683-d5411d989559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:14.451655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756991e5-ae14-4bfd-b683-d5411d989559 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:14.463695 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1134/2260] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:40:14.467948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289ebdf-9445-4519-8604-4d48e56712c4 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:14.478617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289ebdf-9445-4519-8604-4d48e56712c4 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:14.484051 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a1681-6efa-49f3-9d99-198298403a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:14.484757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a1681-6efa-49f3-9d99-198298403a98 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:14.485040 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a1681-6efa-49f3-9d99-198298403a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:14.485370 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a1681-6efa-49f3-9d99-198298403a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:14.486169 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a1681-6efa-49f3-9d99-198298403a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:14.488013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a1681-6efa-49f3-9d99-198298403a98 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:14.503969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1127/2261] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:40:14.505295 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756991e5-ae14-4bfd-b683-d5411d989559 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:14.532685 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756991e5-ae14-4bfd-b683-d5411d989559 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:14.538497 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ac6fa-9003-4f5f-b637-1520bf99b29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:14.538708 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ac6fa-9003-4f5f-b637-1520bf99b29f admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:14.539244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ac6fa-9003-4f5f-b637-1520bf99b29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:14.539521 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ac6fa-9003-4f5f-b637-1520bf99b29f admin admin] PATH_INFO: `/v3/projects/b094932771b5414f90f863f844161030/users/717e1df58f404fa6b12b3f614c9425f4/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:14.556989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ac6fa-9003-4f5f-b637-1520bf99b29f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b094932771b5414f90f863f844161030, user_id=717e1df58f404fa6b12b3f614c9425f4, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:14.565187 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ac6fa-9003-4f5f-b637-1520bf99b29f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:14.579212 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1135/2262] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:40:14 2022] PUT /identity/v3/projects/b094932771b5414f90f863f844161030/users/717e1df58f404fa6b12b3f614c9425f4/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:14.591353 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518f54d-3931-46da-946e-89927d0a3fdf None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:14.591641 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518f54d-3931-46da-946e-89927d0a3fdf None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:14.591981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518f54d-3931-46da-946e-89927d0a3fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:14.601575 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:14.641749 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-5518f54d-3931-46da-946e-89927d0a3fdf None None] MFA Rules not processed for user `717e1df58f404fa6b12b3f614c9425f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:14.643394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-5518f54d-3931-46da-946e-89927d0a3fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:14.873431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1128/2263] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:14.889230 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ac6fa-9003-4f5f-b637-1520bf99b29f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:14.890927 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5b0ac6fa-9003-4f5f-b637-1520bf99b29f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:14.985371 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ac6fa-9003-4f5f-b637-1520bf99b29f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:14.993120 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa63827-0389-4e32-9e5b-dcc9f6aa2c11 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a67e007fc314b34aac2471a7c20f240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:14.994324 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa63827-0389-4e32-9e5b-dcc9f6aa2c11 service glance] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:14.994644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa63827-0389-4e32-9e5b-dcc9f6aa2c11 service glance] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:14.994954 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa63827-0389-4e32-9e5b-dcc9f6aa2c11 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:15.004339 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa63827-0389-4e32-9e5b-dcc9f6aa2c11 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:15.005799 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa63827-0389-4e32-9e5b-dcc9f6aa2c11 service glance] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:15.024990 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1136/2264] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:40:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:40:15.843261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518f54d-3931-46da-946e-89927d0a3fdf None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:15.845147 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-5518f54d-3931-46da-946e-89927d0a3fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:15.927230 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518f54d-3931-46da-946e-89927d0a3fdf None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:15.935063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa0fa6-0e19-4aa2-a5fc-540c0f7715cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:15.935995 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa0fa6-0e19-4aa2-a5fc-540c0f7715cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:15.936251 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa0fa6-0e19-4aa2-a5fc-540c0f7715cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:15.936526 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa0fa6-0e19-4aa2-a5fc-540c0f7715cb admin admin] PATH_INFO: `/v3/users/2b3636afff90484caea7d1bcc0f1e801` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:15.937726 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa0fa6-0e19-4aa2-a5fc-540c0f7715cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b3636afff90484caea7d1bcc0f1e801)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:15.939635 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa0fa6-0e19-4aa2-a5fc-540c0f7715cb admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:16.045948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-69aa0fa6-0e19-4aa2-a5fc-540c0f7715cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b3636afff90484caea7d1bcc0f1e801'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:16.062874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-69aa0fa6-0e19-4aa2-a5fc-540c0f7715cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b3636afff90484caea7d1bcc0f1e801'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:16.070793 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-69aa0fa6-0e19-4aa2-a5fc-540c0f7715cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b3636afff90484caea7d1bcc0f1e801'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:16.071587 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-69aa0fa6-0e19-4aa2-a5fc-540c0f7715cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b3636afff90484caea7d1bcc0f1e801'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:16.089900 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-69aa0fa6-0e19-4aa2-a5fc-540c0f7715cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b3636afff90484caea7d1bcc0f1e801'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:16.096302 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1129/2265] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:40:15 2022] DELETE /identity/v3/users/2b3636afff90484caea7d1bcc0f1e801 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:16.100807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733183ad-a36f-4f7f-b445-7aa84082e553 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:16.101323 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733183ad-a36f-4f7f-b445-7aa84082e553 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:16.101425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733183ad-a36f-4f7f-b445-7aa84082e553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:16.120866 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-733183ad-a36f-4f7f-b445-7aa84082e553 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:16.122163 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-733183ad-a36f-4f7f-b445-7aa84082e553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:16.203938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1137/2266] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:16.221155 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa0fa6-0e19-4aa2-a5fc-540c0f7715cb admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:16.222918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-69aa0fa6-0e19-4aa2-a5fc-540c0f7715cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:16.250764 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:16.323489 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa0fa6-0e19-4aa2-a5fc-540c0f7715cb admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:16.328606 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a6e48-38a1-47cf-90be-bdf90456ddf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:16.329319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a6e48-38a1-47cf-90be-bdf90456ddf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:16.329607 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a6e48-38a1-47cf-90be-bdf90456ddf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:16.329881 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a6e48-38a1-47cf-90be-bdf90456ddf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:16.346921 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a6e48-38a1-47cf-90be-bdf90456ddf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:16.349012 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a6e48-38a1-47cf-90be-bdf90456ddf1 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:16.374217 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1130/2267] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:40:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:40:16.731240 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733183ad-a36f-4f7f-b445-7aa84082e553 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:16.732985 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-733183ad-a36f-4f7f-b445-7aa84082e553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:16.748348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733183ad-a36f-4f7f-b445-7aa84082e553 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:16.766059 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5bc8e-e46f-4666-b09b-b5239201c308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:16.767044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5bc8e-e46f-4666-b09b-b5239201c308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:16.767549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5bc8e-e46f-4666-b09b-b5239201c308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:16.767945 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5bc8e-e46f-4666-b09b-b5239201c308 admin admin] PATH_INFO: `/v3/projects/3d90957df61d421297f92954f9bd3ed4` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:16.769811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5bc8e-e46f-4666-b09b-b5239201c308 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d90957df61d421297f92954f9bd3ed4)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:16.772125 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5bc8e-e46f-4666-b09b-b5239201c308 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:16.798855 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:16.842185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-82b5bc8e-e46f-4666-b09b-b5239201c308 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d90957df61d421297f92954f9bd3ed4', 'request_id': 'req-82b5bc8e-e46f-4666-b09b-b5239201c308'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:16.851463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-82b5bc8e-e46f-4666-b09b-b5239201c308 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d90957df61d421297f92954f9bd3ed4', 'request_id': 'req-82b5bc8e-e46f-4666-b09b-b5239201c308'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:16.860821 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-82b5bc8e-e46f-4666-b09b-b5239201c308 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d90957df61d421297f92954f9bd3ed4', 'request_id': 'req-82b5bc8e-e46f-4666-b09b-b5239201c308'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:16.874915 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-82b5bc8e-e46f-4666-b09b-b5239201c308 admin admin] The token cache is being invalidate because project 3d90957df61d421297f92954f9bd3ed4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:40:16.875251 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-82b5bc8e-e46f-4666-b09b-b5239201c308 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:16.876297 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-82b5bc8e-e46f-4666-b09b-b5239201c308 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:16.878305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1138/2268] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:40:16 2022] DELETE /identity/v3/projects/3d90957df61d421297f92954f9bd3ed4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:16.883090 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a6e48-38a1-47cf-90be-bdf90456ddf1 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:16.907475 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a6e48-38a1-47cf-90be-bdf90456ddf1 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:16.998437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907def2-515e-4c97-9527-1d725dbfd239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:17.001211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907def2-515e-4c97-9527-1d725dbfd239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:17.001211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907def2-515e-4c97-9527-1d725dbfd239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:17.001211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907def2-515e-4c97-9527-1d725dbfd239 admin admin] PATH_INFO: `/v3/users/2211508a748c4dcd828bdcfb19a1214f` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:17.004347 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907def2-515e-4c97-9527-1d725dbfd239 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2211508a748c4dcd828bdcfb19a1214f)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:17.009803 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907def2-515e-4c97-9527-1d725dbfd239 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:17.128715 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b907def2-515e-4c97-9527-1d725dbfd239 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2211508a748c4dcd828bdcfb19a1214f'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:17.139103 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b907def2-515e-4c97-9527-1d725dbfd239 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2211508a748c4dcd828bdcfb19a1214f'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:17.149588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b907def2-515e-4c97-9527-1d725dbfd239 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2211508a748c4dcd828bdcfb19a1214f'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:17.150265 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b907def2-515e-4c97-9527-1d725dbfd239 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2211508a748c4dcd828bdcfb19a1214f'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:17.165945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b907def2-515e-4c97-9527-1d725dbfd239 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2211508a748c4dcd828bdcfb19a1214f'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:17.172947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1131/2269] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:40:16 2022] DELETE /identity/v3/users/2211508a748c4dcd828bdcfb19a1214f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:40:17.658422 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5bc8e-e46f-4666-b09b-b5239201c308 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:17.660691 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-82b5bc8e-e46f-4666-b09b-b5239201c308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:17.689281 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5bc8e-e46f-4666-b09b-b5239201c308 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:17.815890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a32cb-e264-4dd7-a4d7-356897d81fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:17.815890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a32cb-e264-4dd7-a4d7-356897d81fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:17.815890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a32cb-e264-4dd7-a4d7-356897d81fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:17.815890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a32cb-e264-4dd7-a4d7-356897d81fa4 admin admin] PATH_INFO: `/v3/projects/9b75596c509b485181d77aed43ac114d` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:17.816216 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a32cb-e264-4dd7-a4d7-356897d81fa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b75596c509b485181d77aed43ac114d)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:17.816835 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a32cb-e264-4dd7-a4d7-356897d81fa4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:17.889596 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-544a32cb-e264-4dd7-a4d7-356897d81fa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b75596c509b485181d77aed43ac114d', 'request_id': 'req-544a32cb-e264-4dd7-a4d7-356897d81fa4'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:17.898238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-544a32cb-e264-4dd7-a4d7-356897d81fa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b75596c509b485181d77aed43ac114d', 'request_id': 'req-544a32cb-e264-4dd7-a4d7-356897d81fa4'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:17.910450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-544a32cb-e264-4dd7-a4d7-356897d81fa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b75596c509b485181d77aed43ac114d', 'request_id': 'req-544a32cb-e264-4dd7-a4d7-356897d81fa4'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:17.922966 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-544a32cb-e264-4dd7-a4d7-356897d81fa4 admin admin] The token cache is being invalidate because project 9b75596c509b485181d77aed43ac114d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:40:17.923297 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-544a32cb-e264-4dd7-a4d7-356897d81fa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:17.924021 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-544a32cb-e264-4dd7-a4d7-356897d81fa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:17.925555 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1139/2270] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:40:17 2022] DELETE /identity/v3/projects/9b75596c509b485181d77aed43ac114d => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:17.932402 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fe595-db47-4569-a91e-7311b64368e0 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:17.932882 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fe595-db47-4569-a91e-7311b64368e0 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:17.932979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fe595-db47-4569-a91e-7311b64368e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:17.950976 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-442fe595-db47-4569-a91e-7311b64368e0 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:17.952248 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-442fe595-db47-4569-a91e-7311b64368e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:17.994983 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:18.030124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1132/2271] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:18.037766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da575f-cdb1-480b-b30d-6b7d8431a923 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:18.038101 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da575f-cdb1-480b-b30d-6b7d8431a923 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:18.038493 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da575f-cdb1-480b-b30d-6b7d8431a923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:18.058501 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-d5da575f-cdb1-480b-b30d-6b7d8431a923 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:18.060301 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d5da575f-cdb1-480b-b30d-6b7d8431a923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:18.072946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1140/2272] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:18.080277 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fe595-db47-4569-a91e-7311b64368e0 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:18.099420 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fe595-db47-4569-a91e-7311b64368e0 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:18.105039 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b4842-285d-44dc-a57c-aed7a0e54bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:18.105703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b4842-285d-44dc-a57c-aed7a0e54bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:18.105980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b4842-285d-44dc-a57c-aed7a0e54bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:18.106247 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b4842-285d-44dc-a57c-aed7a0e54bc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:18.106975 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b4842-285d-44dc-a57c-aed7a0e54bc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:40:18.107458 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b4842-285d-44dc-a57c-aed7a0e54bc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:18.109135 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b4842-285d-44dc-a57c-aed7a0e54bc3 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:18.125200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1133/2273] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:40:18.129485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da575f-cdb1-480b-b30d-6b7d8431a923 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:18.139360 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da575f-cdb1-480b-b30d-6b7d8431a923 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:18.144422 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d7736-c070-4dcf-912e-d9d054591f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:18.145082 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d7736-c070-4dcf-912e-d9d054591f76 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:18.145398 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d7736-c070-4dcf-912e-d9d054591f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:18.145706 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d7736-c070-4dcf-912e-d9d054591f76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:18.146425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d7736-c070-4dcf-912e-d9d054591f76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:18.148412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d7736-c070-4dcf-912e-d9d054591f76 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:18.176965 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1141/2274] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:40:18 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:40:18.181962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b4842-285d-44dc-a57c-aed7a0e54bc3 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:18.191756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b4842-285d-44dc-a57c-aed7a0e54bc3 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:18.251234 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a26df-7a10-45f9-9e6d-b62b3441d1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:18.251958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a26df-7a10-45f9-9e6d-b62b3441d1b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:18.252241 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a26df-7a10-45f9-9e6d-b62b3441d1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:18.252514 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a26df-7a10-45f9-9e6d-b62b3441d1b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:18.253212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a26df-7a10-45f9-9e6d-b62b3441d1b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:18.255115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a26df-7a10-45f9-9e6d-b62b3441d1b4 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:18.275429 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1134/2275] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:40:18 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:40:18.281346 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d7736-c070-4dcf-912e-d9d054591f76 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:18.291816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d7736-c070-4dcf-912e-d9d054591f76 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:18.296905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9e63d-332f-4b74-bb02-879359039bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:18.297606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9e63d-332f-4b74-bb02-879359039bfe admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:18.297911 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9e63d-332f-4b74-bb02-879359039bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:18.298223 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9e63d-332f-4b74-bb02-879359039bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:18.298999 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d9e63d-332f-4b74-bb02-879359039bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:18.300764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d9e63d-332f-4b74-bb02-879359039bfe admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:18.302186 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:18.313826 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1142/2276] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:40:18.317959 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a26df-7a10-45f9-9e6d-b62b3441d1b4 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:18.328881 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a26df-7a10-45f9-9e6d-b62b3441d1b4 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:18.333629 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf9f29-2db7-4919-9eb1-ac9a0de567f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:18.334488 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf9f29-2db7-4919-9eb1-ac9a0de567f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:18.334771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf9f29-2db7-4919-9eb1-ac9a0de567f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:18.335047 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf9f29-2db7-4919-9eb1-ac9a0de567f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:18.335873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf9f29-2db7-4919-9eb1-ac9a0de567f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:18.337713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf9f29-2db7-4919-9eb1-ac9a0de567f6 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:18.348358 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1135/2277] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:40:18.353183 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9e63d-332f-4b74-bb02-879359039bfe admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:18.363951 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9e63d-332f-4b74-bb02-879359039bfe admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:18.369532 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bc044-d325-42fe-80bd-ebafc6e6cba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:18.370427 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bc044-d325-42fe-80bd-ebafc6e6cba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:18.370746 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bc044-d325-42fe-80bd-ebafc6e6cba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:18.371072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bc044-d325-42fe-80bd-ebafc6e6cba7 admin admin] PATH_INFO: `/v3/projects/66afe6a4a7b14905bb5178530b08b419/users/948682c3549c4974bcfb7549980e0b91/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:18.388490 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bc044-d325-42fe-80bd-ebafc6e6cba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66afe6a4a7b14905bb5178530b08b419, user_id=948682c3549c4974bcfb7549980e0b91, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:18.391615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bc044-d325-42fe-80bd-ebafc6e6cba7 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:18.404951 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1143/2278] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:40:18 2022] PUT /identity/v3/projects/66afe6a4a7b14905bb5178530b08b419/users/948682c3549c4974bcfb7549980e0b91/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:40:18.416608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4826d-75b2-4518-bf18-2f7c8b9d283e None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:18.416897 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4826d-75b2-4518-bf18-2f7c8b9d283e None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:18.417178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4826d-75b2-4518-bf18-2f7c8b9d283e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:18.448716 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-19d4826d-75b2-4518-bf18-2f7c8b9d283e None None] MFA Rules not processed for user `948682c3549c4974bcfb7549980e0b91`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:18.449969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-19d4826d-75b2-4518-bf18-2f7c8b9d283e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:18.521046 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1136/2279] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:18.526247 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bc044-d325-42fe-80bd-ebafc6e6cba7 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:18.595895 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bc044-d325-42fe-80bd-ebafc6e6cba7 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:18.600630 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054cd2ab-eaa2-4e10-b20f-c048e6ad1180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:18.601338 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cd2ab-eaa2-4e10-b20f-c048e6ad1180 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:18.601658 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cd2ab-eaa2-4e10-b20f-c048e6ad1180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:18.601991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cd2ab-eaa2-4e10-b20f-c048e6ad1180 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:18.603409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054cd2ab-eaa2-4e10-b20f-c048e6ad1180 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:18.605408 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054cd2ab-eaa2-4e10-b20f-c048e6ad1180 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:18.636426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1144/2280] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:40:18 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:40:18.637945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4826d-75b2-4518-bf18-2f7c8b9d283e None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:18.650737 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4826d-75b2-4518-bf18-2f7c8b9d283e None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:18.682115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:18.712274 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c2782-2585-4528-8a00-2019adcab1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:18.712946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c2782-2585-4528-8a00-2019adcab1bb admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:18.713230 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c2782-2585-4528-8a00-2019adcab1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:18.713504 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c2782-2585-4528-8a00-2019adcab1bb admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:18.714228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c2782-2585-4528-8a00-2019adcab1bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:18.716234 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c2782-2585-4528-8a00-2019adcab1bb admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:18.736346 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1137/2281] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:40:18 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:40:18.740360 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054cd2ab-eaa2-4e10-b20f-c048e6ad1180 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:18.751369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054cd2ab-eaa2-4e10-b20f-c048e6ad1180 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:18.756737 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469155f6-6095-46db-8403-1858f6803fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:18.757458 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469155f6-6095-46db-8403-1858f6803fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:18.757785 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469155f6-6095-46db-8403-1858f6803fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:18.758104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469155f6-6095-46db-8403-1858f6803fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:18.759568 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469155f6-6095-46db-8403-1858f6803fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:18.761275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469155f6-6095-46db-8403-1858f6803fd6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:18.777045 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1145/2282] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:40:18.778295 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c2782-2585-4528-8a00-2019adcab1bb admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:18.788803 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c2782-2585-4528-8a00-2019adcab1bb admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:18.793693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95afe36-1878-47e0-890a-bc2edbd9a595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:18.794577 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95afe36-1878-47e0-890a-bc2edbd9a595 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:18.794862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95afe36-1878-47e0-890a-bc2edbd9a595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:18.795137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95afe36-1878-47e0-890a-bc2edbd9a595 admin admin] PATH_INFO: `/v3/projects/efc9991da61241afa70f21dde4dfcb89/users/2b65d25efa91483a80dea1b05bfe2438/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:18.814876 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95afe36-1878-47e0-890a-bc2edbd9a595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc9991da61241afa70f21dde4dfcb89, user_id=2b65d25efa91483a80dea1b05bfe2438, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:18.817748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95afe36-1878-47e0-890a-bc2edbd9a595 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:18.833391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1138/2283] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:40:18 2022] PUT /identity/v3/projects/efc9991da61241afa70f21dde4dfcb89/users/2b65d25efa91483a80dea1b05bfe2438/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:18.838551 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469155f6-6095-46db-8403-1858f6803fd6 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:18.848460 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469155f6-6095-46db-8403-1858f6803fd6 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:18.916073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab94952-9f1a-4e04-b035-927ad0adc8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:18.916073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab94952-9f1a-4e04-b035-927ad0adc8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:18.916073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab94952-9f1a-4e04-b035-927ad0adc8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:18.916073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab94952-9f1a-4e04-b035-927ad0adc8c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:18.916368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab94952-9f1a-4e04-b035-927ad0adc8c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:18.916368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab94952-9f1a-4e04-b035-927ad0adc8c4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:18.928457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1146/2284] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:40:18.933069 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95afe36-1878-47e0-890a-bc2edbd9a595 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:18.944338 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95afe36-1878-47e0-890a-bc2edbd9a595 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:18.949865 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40378e7b-ae2e-4f42-afe3-8641e0c43ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:18.950884 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40378e7b-ae2e-4f42-afe3-8641e0c43ac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:18.951236 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40378e7b-ae2e-4f42-afe3-8641e0c43ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:18.951581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40378e7b-ae2e-4f42-afe3-8641e0c43ac4 admin admin] PATH_INFO: `/v3/projects/efc9991da61241afa70f21dde4dfcb89/users/2b65d25efa91483a80dea1b05bfe2438/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:18.953830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40378e7b-ae2e-4f42-afe3-8641e0c43ac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc9991da61241afa70f21dde4dfcb89, user_id=2b65d25efa91483a80dea1b05bfe2438, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:18.956866 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40378e7b-ae2e-4f42-afe3-8641e0c43ac4 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:18.968642 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1139/2285] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:40:18 2022] PUT /identity/v3/projects/efc9991da61241afa70f21dde4dfcb89/users/2b65d25efa91483a80dea1b05bfe2438/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:18.978844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169be22-f6e6-4e8b-a882-db45fb6bdb19 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:18.979255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169be22-f6e6-4e8b-a882-db45fb6bdb19 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:18.979544 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169be22-f6e6-4e8b-a882-db45fb6bdb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:19.017276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3169be22-f6e6-4e8b-a882-db45fb6bdb19 None None] MFA Rules not processed for user `2b65d25efa91483a80dea1b05bfe2438`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:19.018482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3169be22-f6e6-4e8b-a882-db45fb6bdb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:19.057136 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:19.104305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1147/2286] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:19.302287 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40378e7b-ae2e-4f42-afe3-8641e0c43ac4 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:19.304354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-40378e7b-ae2e-4f42-afe3-8641e0c43ac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:19.375739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40378e7b-ae2e-4f42-afe3-8641e0c43ac4 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:19.391644 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1bde3-8fd9-418c-8000-ef8a7554d795 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:19.392428 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1bde3-8fd9-418c-8000-ef8a7554d795 service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:19.392689 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1bde3-8fd9-418c-8000-ef8a7554d795 service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:19.392966 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1bde3-8fd9-418c-8000-ef8a7554d795 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:19.457540 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1bde3-8fd9-418c-8000-ef8a7554d795 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:19.458974 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1bde3-8fd9-418c-8000-ef8a7554d795 service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:19.468368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1140/2287] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:40:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:40:23.866649 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169be22-f6e6-4e8b-a882-db45fb6bdb19 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:23.876523 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169be22-f6e6-4e8b-a882-db45fb6bdb19 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:23.880125 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ba94d-0d80-403b-9455-05d1a78ca595 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:23.880757 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ba94d-0d80-403b-9455-05d1a78ca595 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:23.881065 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ba94d-0d80-403b-9455-05d1a78ca595 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:23.881344 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ba94d-0d80-403b-9455-05d1a78ca595 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:23.890917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37ba94d-0d80-403b-9455-05d1a78ca595 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:23.892447 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37ba94d-0d80-403b-9455-05d1a78ca595 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:23.902718 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1148/2288] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:40:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:40:24.169935 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1bde3-8fd9-418c-8000-ef8a7554d795 service cinder] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:24.172804 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-59a1bde3-8fd9-418c-8000-ef8a7554d795 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:24.244338 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:24.297007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1bde3-8fd9-418c-8000-ef8a7554d795 service cinder] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:24.302628 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32255347-57c4-47e7-9cd4-90cf705ca6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:24.303564 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32255347-57c4-47e7-9cd4-90cf705ca6d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:24.303924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32255347-57c4-47e7-9cd4-90cf705ca6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:24.304215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32255347-57c4-47e7-9cd4-90cf705ca6d3 admin admin] PATH_INFO: `/v3/users/812c09be835944fc85c041ae93f990cd` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:24.306707 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32255347-57c4-47e7-9cd4-90cf705ca6d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=812c09be835944fc85c041ae93f990cd)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:24.308167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32255347-57c4-47e7-9cd4-90cf705ca6d3 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:24.364587 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-32255347-57c4-47e7-9cd4-90cf705ca6d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '812c09be835944fc85c041ae93f990cd'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:24.373126 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-32255347-57c4-47e7-9cd4-90cf705ca6d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '812c09be835944fc85c041ae93f990cd'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:24.389390 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-32255347-57c4-47e7-9cd4-90cf705ca6d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '812c09be835944fc85c041ae93f990cd'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:24.390243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-32255347-57c4-47e7-9cd4-90cf705ca6d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '812c09be835944fc85c041ae93f990cd'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:24.409462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-32255347-57c4-47e7-9cd4-90cf705ca6d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '812c09be835944fc85c041ae93f990cd'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:24.417391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1141/2289] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:40:24 2022] DELETE /identity/v3/users/812c09be835944fc85c041ae93f990cd => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:24.422011 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73dbd0-4808-4eaa-8461-91a2962f6197 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:24.422239 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73dbd0-4808-4eaa-8461-91a2962f6197 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:24.422544 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73dbd0-4808-4eaa-8461-91a2962f6197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:24.440895 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9f73dbd0-4808-4eaa-8461-91a2962f6197 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:24.442088 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9f73dbd0-4808-4eaa-8461-91a2962f6197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:24.514048 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1149/2290] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:24.524038 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32255347-57c4-47e7-9cd4-90cf705ca6d3 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:24.525391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-32255347-57c4-47e7-9cd4-90cf705ca6d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:24.587267 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32255347-57c4-47e7-9cd4-90cf705ca6d3 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:24.591109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf697e21-3038-46af-a405-d30f8856a9e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:24.591754 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf697e21-3038-46af-a405-d30f8856a9e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:24.592128 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf697e21-3038-46af-a405-d30f8856a9e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:24.592415 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf697e21-3038-46af-a405-d30f8856a9e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:24.603467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf697e21-3038-46af-a405-d30f8856a9e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:24.605019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf697e21-3038-46af-a405-d30f8856a9e1 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:24.617105 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1142/2291] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:40:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:40:24.932473 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73dbd0-4808-4eaa-8461-91a2962f6197 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:24.934079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9f73dbd0-4808-4eaa-8461-91a2962f6197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:24.948733 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73dbd0-4808-4eaa-8461-91a2962f6197 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:24.954212 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac457aea-058f-4f4a-99fb-2cbf361187f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:24.955070 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac457aea-058f-4f4a-99fb-2cbf361187f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:24.955368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac457aea-058f-4f4a-99fb-2cbf361187f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:24.955941 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac457aea-058f-4f4a-99fb-2cbf361187f1 admin admin] PATH_INFO: `/v3/projects/c2e8f3ca5d224847a240caa5a090b025` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:24.957165 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac457aea-058f-4f4a-99fb-2cbf361187f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2e8f3ca5d224847a240caa5a090b025)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:24.959276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac457aea-058f-4f4a-99fb-2cbf361187f1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:25.014293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ac457aea-058f-4f4a-99fb-2cbf361187f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2e8f3ca5d224847a240caa5a090b025', 'request_id': 'req-ac457aea-058f-4f4a-99fb-2cbf361187f1'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:25.023211 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ac457aea-058f-4f4a-99fb-2cbf361187f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2e8f3ca5d224847a240caa5a090b025', 'request_id': 'req-ac457aea-058f-4f4a-99fb-2cbf361187f1'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:25.030050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ac457aea-058f-4f4a-99fb-2cbf361187f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2e8f3ca5d224847a240caa5a090b025', 'request_id': 'req-ac457aea-058f-4f4a-99fb-2cbf361187f1'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:25.042602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ac457aea-058f-4f4a-99fb-2cbf361187f1 admin admin] The token cache is being invalidate because project c2e8f3ca5d224847a240caa5a090b025 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:40:25.042892 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ac457aea-058f-4f4a-99fb-2cbf361187f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:25.043544 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ac457aea-058f-4f4a-99fb-2cbf361187f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:25.045665 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1150/2292] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:40:24 2022] DELETE /identity/v3/projects/c2e8f3ca5d224847a240caa5a090b025 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:30.244966 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4717894-9665-4180-a51a-9e99553d8e64 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:30.245455 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4717894-9665-4180-a51a-9e99553d8e64 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:30.245613 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4717894-9665-4180-a51a-9e99553d8e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:30.277542 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-b4717894-9665-4180-a51a-9e99553d8e64 None None] MFA Rules not processed for user `e9e358afe0974c0c8bd53104ebbff445`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:30.277732 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-b4717894-9665-4180-a51a-9e99553d8e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:30.329209 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:30.341635 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1143/2293] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jun 6 10:40:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:30.395379 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac457aea-058f-4f4a-99fb-2cbf361187f1 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:30.397213 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ac457aea-058f-4f4a-99fb-2cbf361187f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:30.444642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:30.474234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac457aea-058f-4f4a-99fb-2cbf361187f1 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:30.480164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00bb04-16a8-4a65-887c-c550ef744cdb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fede9e52ad274285b09d748a684ede38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:30.480821 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00bb04-16a8-4a65-887c-c550ef744cdb service swift] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:30.481103 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00bb04-16a8-4a65-887c-c550ef744cdb service swift] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:30.481377 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00bb04-16a8-4a65-887c-c550ef744cdb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:30.489836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00bb04-16a8-4a65-887c-c550ef744cdb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:30.491412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00bb04-16a8-4a65-887c-c550ef744cdb service swift] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:30.499363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1151/2294] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jun 6 10:40:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 10:40:33.178419 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4717894-9665-4180-a51a-9e99553d8e64 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:33.180846 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-b4717894-9665-4180-a51a-9e99553d8e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:33.275465 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4717894-9665-4180-a51a-9e99553d8e64 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:33.280561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810eed3-f571-465c-b624-839f54f18e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:33.281539 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810eed3-f571-465c-b624-839f54f18e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:33.281900 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810eed3-f571-465c-b624-839f54f18e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:33.282277 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810eed3-f571-465c-b624-839f54f18e2b admin admin] PATH_INFO: `/v3/users/948682c3549c4974bcfb7549980e0b91` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:33.283566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e810eed3-f571-465c-b624-839f54f18e2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=948682c3549c4974bcfb7549980e0b91)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:33.285684 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e810eed3-f571-465c-b624-839f54f18e2b admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:33.373698 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e810eed3-f571-465c-b624-839f54f18e2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '948682c3549c4974bcfb7549980e0b91'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:33.408238 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e810eed3-f571-465c-b624-839f54f18e2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '948682c3549c4974bcfb7549980e0b91'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:33.428738 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e810eed3-f571-465c-b624-839f54f18e2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '948682c3549c4974bcfb7549980e0b91'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:33.430678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e810eed3-f571-465c-b624-839f54f18e2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '948682c3549c4974bcfb7549980e0b91'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:33.453739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e810eed3-f571-465c-b624-839f54f18e2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '948682c3549c4974bcfb7549980e0b91'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:33.463260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1144/2295] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:40:33 2022] DELETE /identity/v3/users/948682c3549c4974bcfb7549980e0b91 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:40:33.468046 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7966480-2402-42bc-bef2-09cc40cf208e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:33.468642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7966480-2402-42bc-bef2-09cc40cf208e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:33.468806 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7966480-2402-42bc-bef2-09cc40cf208e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:33.490439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-e7966480-2402-42bc-bef2-09cc40cf208e None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:33.491640 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e7966480-2402-42bc-bef2-09cc40cf208e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:33.555074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1152/2296] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:33.564787 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810eed3-f571-465c-b624-839f54f18e2b admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:33.566306 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-e810eed3-f571-465c-b624-839f54f18e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:33.633202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810eed3-f571-465c-b624-839f54f18e2b admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:33.636920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d01dcb-340b-46ee-9d12-d9638f9241d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:33.637562 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d01dcb-340b-46ee-9d12-d9638f9241d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:33.637850 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d01dcb-340b-46ee-9d12-d9638f9241d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:33.638118 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d01dcb-340b-46ee-9d12-d9638f9241d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:33.647489 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d01dcb-340b-46ee-9d12-d9638f9241d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:33.649321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d01dcb-340b-46ee-9d12-d9638f9241d5 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:33.658876 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1145/2297] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:40:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:40:33.980342 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7966480-2402-42bc-bef2-09cc40cf208e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:33.981953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e7966480-2402-42bc-bef2-09cc40cf208e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:34.011914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7966480-2402-42bc-bef2-09cc40cf208e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:34.017085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd60db-e126-4c8c-97e5-f859f99c2cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:34.017952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd60db-e126-4c8c-97e5-f859f99c2cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:34.018241 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd60db-e126-4c8c-97e5-f859f99c2cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:34.018503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd60db-e126-4c8c-97e5-f859f99c2cce admin admin] PATH_INFO: `/v3/projects/66afe6a4a7b14905bb5178530b08b419` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:34.020478 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd60db-e126-4c8c-97e5-f859f99c2cce admin admin] RBAC: Authorizing `identity:delete_project(project_id=66afe6a4a7b14905bb5178530b08b419)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:34.022669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd60db-e126-4c8c-97e5-f859f99c2cce admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:34.082604 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c0cd60db-e126-4c8c-97e5-f859f99c2cce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66afe6a4a7b14905bb5178530b08b419', 'request_id': 'req-c0cd60db-e126-4c8c-97e5-f859f99c2cce'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:34.091674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c0cd60db-e126-4c8c-97e5-f859f99c2cce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66afe6a4a7b14905bb5178530b08b419', 'request_id': 'req-c0cd60db-e126-4c8c-97e5-f859f99c2cce'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:34.102313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c0cd60db-e126-4c8c-97e5-f859f99c2cce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66afe6a4a7b14905bb5178530b08b419', 'request_id': 'req-c0cd60db-e126-4c8c-97e5-f859f99c2cce'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:34.114791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c0cd60db-e126-4c8c-97e5-f859f99c2cce admin admin] The token cache is being invalidate because project 66afe6a4a7b14905bb5178530b08b419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:40:34.115317 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c0cd60db-e126-4c8c-97e5-f859f99c2cce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:34.116045 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c0cd60db-e126-4c8c-97e5-f859f99c2cce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:34.117741 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1153/2298] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:40:33 2022] DELETE /identity/v3/projects/66afe6a4a7b14905bb5178530b08b419 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:34.122332 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d01dcb-340b-46ee-9d12-d9638f9241d5 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:34.135189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d01dcb-340b-46ee-9d12-d9638f9241d5 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:34.151935 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:34.215401 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac3a64-07e9-46ee-a7b6-0ead3a43a436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:34.216329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac3a64-07e9-46ee-a7b6-0ead3a43a436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:34.216616 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac3a64-07e9-46ee-a7b6-0ead3a43a436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:34.216887 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac3a64-07e9-46ee-a7b6-0ead3a43a436 admin admin] PATH_INFO: `/v3/users/2b65d25efa91483a80dea1b05bfe2438` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:34.218145 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac3a64-07e9-46ee-a7b6-0ead3a43a436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b65d25efa91483a80dea1b05bfe2438)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:34.220139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac3a64-07e9-46ee-a7b6-0ead3a43a436 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:34.281677 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-76ac3a64-07e9-46ee-a7b6-0ead3a43a436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b65d25efa91483a80dea1b05bfe2438'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:34.289742 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-76ac3a64-07e9-46ee-a7b6-0ead3a43a436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b65d25efa91483a80dea1b05bfe2438'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:34.298774 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-76ac3a64-07e9-46ee-a7b6-0ead3a43a436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b65d25efa91483a80dea1b05bfe2438'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:34.299540 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-76ac3a64-07e9-46ee-a7b6-0ead3a43a436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b65d25efa91483a80dea1b05bfe2438'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:34.318654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-76ac3a64-07e9-46ee-a7b6-0ead3a43a436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b65d25efa91483a80dea1b05bfe2438'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:34.326715 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1146/2299] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:40:34 2022] DELETE /identity/v3/users/2b65d25efa91483a80dea1b05bfe2438 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:34.981157 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd60db-e126-4c8c-97e5-f859f99c2cce admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:34.982645 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c0cd60db-e126-4c8c-97e5-f859f99c2cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:35.003506 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd60db-e126-4c8c-97e5-f859f99c2cce admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:35.099356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:35.126503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045b419-66d9-4684-8aba-fb58fa19e7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:35.127424 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045b419-66d9-4684-8aba-fb58fa19e7a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:35.127721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045b419-66d9-4684-8aba-fb58fa19e7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:35.128027 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045b419-66d9-4684-8aba-fb58fa19e7a9 admin admin] PATH_INFO: `/v3/projects/efc9991da61241afa70f21dde4dfcb89` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:35.136671 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045b419-66d9-4684-8aba-fb58fa19e7a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efc9991da61241afa70f21dde4dfcb89)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:35.138912 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045b419-66d9-4684-8aba-fb58fa19e7a9 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:35.219965 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4045b419-66d9-4684-8aba-fb58fa19e7a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efc9991da61241afa70f21dde4dfcb89', 'request_id': 'req-4045b419-66d9-4684-8aba-fb58fa19e7a9'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:35.229412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4045b419-66d9-4684-8aba-fb58fa19e7a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efc9991da61241afa70f21dde4dfcb89', 'request_id': 'req-4045b419-66d9-4684-8aba-fb58fa19e7a9'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:35.236489 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4045b419-66d9-4684-8aba-fb58fa19e7a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efc9991da61241afa70f21dde4dfcb89', 'request_id': 'req-4045b419-66d9-4684-8aba-fb58fa19e7a9'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:35.256092 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4045b419-66d9-4684-8aba-fb58fa19e7a9 admin admin] The token cache is being invalidate because project efc9991da61241afa70f21dde4dfcb89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:40:35.256370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4045b419-66d9-4684-8aba-fb58fa19e7a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:35.257023 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4045b419-66d9-4684-8aba-fb58fa19e7a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:35.264219 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1154/2300] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:40:34 2022] DELETE /identity/v3/projects/efc9991da61241afa70f21dde4dfcb89 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:35.265755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25594c6c-1921-4cb2-8f3d-dcf0dfc1f31a None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:35.266292 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25594c6c-1921-4cb2-8f3d-dcf0dfc1f31a None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:35.266292 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25594c6c-1921-4cb2-8f3d-dcf0dfc1f31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:35.301683 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-25594c6c-1921-4cb2-8f3d-dcf0dfc1f31a None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:35.303151 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-25594c6c-1921-4cb2-8f3d-dcf0dfc1f31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:35.421942 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1147/2301] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:40:35.432947 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9280a0a-4609-4eee-a9f2-40308f12c4a4 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:35.433470 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9280a0a-4609-4eee-a9f2-40308f12c4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:35.433749 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9280a0a-4609-4eee-a9f2-40308f12c4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:35.463660 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b9280a0a-4609-4eee-a9f2-40308f12c4a4 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:35.467831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b9280a0a-4609-4eee-a9f2-40308f12c4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:35.483767 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1155/2302] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:35.488145 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25594c6c-1921-4cb2-8f3d-dcf0dfc1f31a None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:35.502346 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25594c6c-1921-4cb2-8f3d-dcf0dfc1f31a None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:35.512989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b66440-96f6-4fd2-88ac-e6c83e407055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:35.513898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b66440-96f6-4fd2-88ac-e6c83e407055 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:35.514334 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b66440-96f6-4fd2-88ac-e6c83e407055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:35.514748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b66440-96f6-4fd2-88ac-e6c83e407055 admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:35.515672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b66440-96f6-4fd2-88ac-e6c83e407055 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:40:35.516178 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b66440-96f6-4fd2-88ac-e6c83e407055 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:35.517808 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b66440-96f6-4fd2-88ac-e6c83e407055 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:35.539950 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1148/2303] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:40:35.540967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9280a0a-4609-4eee-a9f2-40308f12c4a4 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:35.552392 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9280a0a-4609-4eee-a9f2-40308f12c4a4 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:35.558302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a943b6-847d-49f6-99c1-91779142d744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:35.558988 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a943b6-847d-49f6-99c1-91779142d744 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:35.559266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a943b6-847d-49f6-99c1-91779142d744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:35.559568 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a943b6-847d-49f6-99c1-91779142d744 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:35.560459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a943b6-847d-49f6-99c1-91779142d744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:35.562194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a943b6-847d-49f6-99c1-91779142d744 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:35.599273 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1156/2304] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:40:35 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:40:35.603023 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b66440-96f6-4fd2-88ac-e6c83e407055 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:35.613487 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b66440-96f6-4fd2-88ac-e6c83e407055 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:35.630181 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:35.777957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56326f66-9b6c-4cc1-b1a1-dff7e0e78007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:35.779159 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56326f66-9b6c-4cc1-b1a1-dff7e0e78007 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:35.779691 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56326f66-9b6c-4cc1-b1a1-dff7e0e78007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:35.780181 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56326f66-9b6c-4cc1-b1a1-dff7e0e78007 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:35.781627 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56326f66-9b6c-4cc1-b1a1-dff7e0e78007 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:35.784149 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56326f66-9b6c-4cc1-b1a1-dff7e0e78007 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:35.859837 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1149/2305] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:40:35 2022] POST /identity/v3/users => generated 292 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:40:35.865269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a943b6-847d-49f6-99c1-91779142d744 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:35.891141 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a943b6-847d-49f6-99c1-91779142d744 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:35.900360 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119c5b7-6c81-4ee2-90e3-c98664f92a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:35.901865 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119c5b7-6c81-4ee2-90e3-c98664f92a77 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:35.902773 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119c5b7-6c81-4ee2-90e3-c98664f92a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:35.903225 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119c5b7-6c81-4ee2-90e3-c98664f92a77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:35.904190 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119c5b7-6c81-4ee2-90e3-c98664f92a77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:35.907644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119c5b7-6c81-4ee2-90e3-c98664f92a77 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:35.928460 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1157/2306] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:40:35.933367 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56326f66-9b6c-4cc1-b1a1-dff7e0e78007 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:35.946226 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56326f66-9b6c-4cc1-b1a1-dff7e0e78007 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:35.983282 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3b8ba-0045-4657-a6f0-7e0a09587436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:35.984138 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3b8ba-0045-4657-a6f0-7e0a09587436 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:35.984520 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3b8ba-0045-4657-a6f0-7e0a09587436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:35.984894 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3b8ba-0045-4657-a6f0-7e0a09587436 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:35.985732 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3b8ba-0045-4657-a6f0-7e0a09587436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:35.988240 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3b8ba-0045-4657-a6f0-7e0a09587436 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:36.026691 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1150/2307] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:40:36.030696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119c5b7-6c81-4ee2-90e3-c98664f92a77 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:36.055394 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119c5b7-6c81-4ee2-90e3-c98664f92a77 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:36.062676 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6753a6-7200-40ab-a2f3-f91024cfe5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:36.064104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6753a6-7200-40ab-a2f3-f91024cfe5ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:36.064521 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6753a6-7200-40ab-a2f3-f91024cfe5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:36.064978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6753a6-7200-40ab-a2f3-f91024cfe5ec admin admin] PATH_INFO: `/v3/projects/a81deb26aaca4004bfab9bedb44d9e8d/users/c92b27a18c9c4e37bf19f3454be020bf/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:36.091313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6753a6-7200-40ab-a2f3-f91024cfe5ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=a81deb26aaca4004bfab9bedb44d9e8d, user_id=c92b27a18c9c4e37bf19f3454be020bf, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:36.095055 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6753a6-7200-40ab-a2f3-f91024cfe5ec admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:36.118966 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1158/2308] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:40:36 2022] PUT /identity/v3/projects/a81deb26aaca4004bfab9bedb44d9e8d/users/c92b27a18c9c4e37bf19f3454be020bf/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:36.129555 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7dc61c-95d6-45ed-9860-727fb5be85b1 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:36.129964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7dc61c-95d6-45ed-9860-727fb5be85b1 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:36.130146 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7dc61c-95d6-45ed-9860-727fb5be85b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:36.176389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-2c7dc61c-95d6-45ed-9860-727fb5be85b1 None None] MFA Rules not processed for user `c92b27a18c9c4e37bf19f3454be020bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:36.178330 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-2c7dc61c-95d6-45ed-9860-727fb5be85b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:36.299886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1151/2309] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:36.305501 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6753a6-7200-40ab-a2f3-f91024cfe5ec admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:36.307591 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5f6753a6-7200-40ab-a2f3-f91024cfe5ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:36.400660 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6753a6-7200-40ab-a2f3-f91024cfe5ec admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:36.404762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb72dc-c450-4402-add4-c9a5da79f101 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:36.405563 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb72dc-c450-4402-add4-c9a5da79f101 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:36.405890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb72dc-c450-4402-add4-c9a5da79f101 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:36.406136 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb72dc-c450-4402-add4-c9a5da79f101 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:36.416205 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb72dc-c450-4402-add4-c9a5da79f101 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:36.417659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb72dc-c450-4402-add4-c9a5da79f101 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:36.458481 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1159/2310] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:40:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:40:36.468266 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7dc61c-95d6-45ed-9860-727fb5be85b1 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:36.561261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7dc61c-95d6-45ed-9860-727fb5be85b1 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:36.566668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685356b8-49d6-48e3-bb3f-7c535e0f441e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:36.567499 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685356b8-49d6-48e3-bb3f-7c535e0f441e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:36.567821 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685356b8-49d6-48e3-bb3f-7c535e0f441e admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:36.568255 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685356b8-49d6-48e3-bb3f-7c535e0f441e admin admin] PATH_INFO: `/v3/users/c92b27a18c9c4e37bf19f3454be020bf` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:36.569816 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685356b8-49d6-48e3-bb3f-7c535e0f441e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c92b27a18c9c4e37bf19f3454be020bf)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:36.573491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685356b8-49d6-48e3-bb3f-7c535e0f441e admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:36.623443 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:36.652854 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-685356b8-49d6-48e3-bb3f-7c535e0f441e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c92b27a18c9c4e37bf19f3454be020bf'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:36.669332 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-685356b8-49d6-48e3-bb3f-7c535e0f441e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c92b27a18c9c4e37bf19f3454be020bf'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:36.680045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-685356b8-49d6-48e3-bb3f-7c535e0f441e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c92b27a18c9c4e37bf19f3454be020bf'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:36.680681 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-685356b8-49d6-48e3-bb3f-7c535e0f441e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c92b27a18c9c4e37bf19f3454be020bf'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:36.700385 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-685356b8-49d6-48e3-bb3f-7c535e0f441e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c92b27a18c9c4e37bf19f3454be020bf'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:36.711661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1152/2311] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:40:36 2022] DELETE /identity/v3/users/c92b27a18c9c4e37bf19f3454be020bf => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:36.717367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adaaff-f2b9-4ae7-adc2-b45052d28ad1 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:36.717759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adaaff-f2b9-4ae7-adc2-b45052d28ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:36.718062 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adaaff-f2b9-4ae7-adc2-b45052d28ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:36.737916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-46adaaff-f2b9-4ae7-adc2-b45052d28ad1 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:36.739542 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-46adaaff-f2b9-4ae7-adc2-b45052d28ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:36.756612 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:36.831283 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1160/2312] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:36.839728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685356b8-49d6-48e3-bb3f-7c535e0f441e admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:36.842900 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-685356b8-49d6-48e3-bb3f-7c535e0f441e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:36.925304 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685356b8-49d6-48e3-bb3f-7c535e0f441e admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:36.929278 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65d8a9-7b45-43a6-90cc-669e7a69e15d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:36.930754 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65d8a9-7b45-43a6-90cc-669e7a69e15d service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:36.931035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65d8a9-7b45-43a6-90cc-669e7a69e15d service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:36.932081 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65d8a9-7b45-43a6-90cc-669e7a69e15d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:36.942566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65d8a9-7b45-43a6-90cc-669e7a69e15d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:36.944366 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65d8a9-7b45-43a6-90cc-669e7a69e15d service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:36.956728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1153/2313] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:40:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:40:37.438143 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46adaaff-f2b9-4ae7-adc2-b45052d28ad1 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:37.441263 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-46adaaff-f2b9-4ae7-adc2-b45052d28ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:37.483473 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46adaaff-f2b9-4ae7-adc2-b45052d28ad1 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:37.501883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2f3d4-6141-49cd-aaae-25548dd80382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:37.506310 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2f3d4-6141-49cd-aaae-25548dd80382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:37.506992 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2f3d4-6141-49cd-aaae-25548dd80382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:37.507905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2f3d4-6141-49cd-aaae-25548dd80382 admin admin] PATH_INFO: `/v3/projects/a81deb26aaca4004bfab9bedb44d9e8d` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:37.514086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2f3d4-6141-49cd-aaae-25548dd80382 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a81deb26aaca4004bfab9bedb44d9e8d)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:37.525310 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2f3d4-6141-49cd-aaae-25548dd80382 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:37.619369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-15b2f3d4-6141-49cd-aaae-25548dd80382 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a81deb26aaca4004bfab9bedb44d9e8d', 'request_id': 'req-15b2f3d4-6141-49cd-aaae-25548dd80382'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:37.633111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-15b2f3d4-6141-49cd-aaae-25548dd80382 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a81deb26aaca4004bfab9bedb44d9e8d', 'request_id': 'req-15b2f3d4-6141-49cd-aaae-25548dd80382'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:37.640407 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-15b2f3d4-6141-49cd-aaae-25548dd80382 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a81deb26aaca4004bfab9bedb44d9e8d', 'request_id': 'req-15b2f3d4-6141-49cd-aaae-25548dd80382'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:37.659336 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-15b2f3d4-6141-49cd-aaae-25548dd80382 admin admin] The token cache is being invalidate because project a81deb26aaca4004bfab9bedb44d9e8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:40:37.659770 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-15b2f3d4-6141-49cd-aaae-25548dd80382 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:37.660541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-15b2f3d4-6141-49cd-aaae-25548dd80382 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:37.667398 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1161/2314] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:40:37 2022] DELETE /identity/v3/projects/a81deb26aaca4004bfab9bedb44d9e8d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:40:37.669358 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c27603-f2b8-4b0d-8d0b-a89538840841 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:37.669953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c27603-f2b8-4b0d-8d0b-a89538840841 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:37.670076 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c27603-f2b8-4b0d-8d0b-a89538840841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:37.691897 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-d6c27603-f2b8-4b0d-8d0b-a89538840841 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:37.693158 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-d6c27603-f2b8-4b0d-8d0b-a89538840841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:37.776133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1154/2315] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:37.786121 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d067c20-75ee-4f10-8db2-101109d9c1e4 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:37.786814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d067c20-75ee-4f10-8db2-101109d9c1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:37.787110 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d067c20-75ee-4f10-8db2-101109d9c1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:37.811107 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2d067c20-75ee-4f10-8db2-101109d9c1e4 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:37.812378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2d067c20-75ee-4f10-8db2-101109d9c1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:37.821826 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1162/2316] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:37.826546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c27603-f2b8-4b0d-8d0b-a89538840841 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:37.841769 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c27603-f2b8-4b0d-8d0b-a89538840841 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:37.849642 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69f4ba-3359-471d-b82a-346d02e88bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:37.850936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69f4ba-3359-471d-b82a-346d02e88bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:37.852256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69f4ba-3359-471d-b82a-346d02e88bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:37.852425 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69f4ba-3359-471d-b82a-346d02e88bc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:37.853674 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b69f4ba-3359-471d-b82a-346d02e88bc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:40:37.854006 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b69f4ba-3359-471d-b82a-346d02e88bc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:37.855662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b69f4ba-3359-471d-b82a-346d02e88bc0 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:37.878606 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1155/2317] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:40:37.880133 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d067c20-75ee-4f10-8db2-101109d9c1e4 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:37.891605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d067c20-75ee-4f10-8db2-101109d9c1e4 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:37.897339 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15456f71-6dbd-4c29-9f45-caedaefa7855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:37.898403 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15456f71-6dbd-4c29-9f45-caedaefa7855 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:37.898649 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15456f71-6dbd-4c29-9f45-caedaefa7855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:37.898838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15456f71-6dbd-4c29-9f45-caedaefa7855 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:37.899644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15456f71-6dbd-4c29-9f45-caedaefa7855 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:37.901643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15456f71-6dbd-4c29-9f45-caedaefa7855 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:37.953205 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1163/2318] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:40:37 2022] POST /identity/v3/projects => generated 391 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:40:37.967628 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69f4ba-3359-471d-b82a-346d02e88bc0 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:37.981478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69f4ba-3359-471d-b82a-346d02e88bc0 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:38.041701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:38.124551 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb65b7-68d6-4a37-8012-d17dfc943db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:38.125560 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb65b7-68d6-4a37-8012-d17dfc943db1 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:38.126013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb65b7-68d6-4a37-8012-d17dfc943db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:38.128194 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb65b7-68d6-4a37-8012-d17dfc943db1 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:38.128194 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb65b7-68d6-4a37-8012-d17dfc943db1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:38.131209 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb65b7-68d6-4a37-8012-d17dfc943db1 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:38.175732 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1156/2319] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:40:37 2022] POST /identity/v3/users => generated 296 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:40:38.180874 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15456f71-6dbd-4c29-9f45-caedaefa7855 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:38.197901 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15456f71-6dbd-4c29-9f45-caedaefa7855 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:38.222712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552c690-4bee-431e-ae0b-1c8ad786bb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:38.223442 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552c690-4bee-431e-ae0b-1c8ad786bb97 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:38.224052 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552c690-4bee-431e-ae0b-1c8ad786bb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:38.224052 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552c690-4bee-431e-ae0b-1c8ad786bb97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:38.224964 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552c690-4bee-431e-ae0b-1c8ad786bb97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:38.233145 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552c690-4bee-431e-ae0b-1c8ad786bb97 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:38.269825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb65b7-68d6-4a37-8012-d17dfc943db1 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:38.276451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1164/2320] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:40:38.286730 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb65b7-68d6-4a37-8012-d17dfc943db1 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:38.306936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b093b8-43c6-4abf-865a-c1abcda64dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:38.307626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b093b8-43c6-4abf-865a-c1abcda64dbb admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:38.307938 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b093b8-43c6-4abf-865a-c1abcda64dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:38.308216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b093b8-43c6-4abf-865a-c1abcda64dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:38.308954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b093b8-43c6-4abf-865a-c1abcda64dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:38.310543 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b093b8-43c6-4abf-865a-c1abcda64dbb admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:38.334014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1157/2321] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:40:38.338306 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552c690-4bee-431e-ae0b-1c8ad786bb97 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:38.353887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552c690-4bee-431e-ae0b-1c8ad786bb97 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:38.376542 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc93bc9-1ed7-4c6c-b994-873a64d910af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:38.380218 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc93bc9-1ed7-4c6c-b994-873a64d910af admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:38.380218 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc93bc9-1ed7-4c6c-b994-873a64d910af admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:38.380218 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc93bc9-1ed7-4c6c-b994-873a64d910af admin admin] PATH_INFO: `/v3/projects/4e6700296b224ffa849326518a611ee2/users/ef1449e1ec394177ba675fdc1c9bccbd/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:38.405852 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc93bc9-1ed7-4c6c-b994-873a64d910af admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e6700296b224ffa849326518a611ee2, user_id=ef1449e1ec394177ba675fdc1c9bccbd, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:38.409109 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc93bc9-1ed7-4c6c-b994-873a64d910af admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:38.431285 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1165/2322] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:40:38 2022] PUT /identity/v3/projects/4e6700296b224ffa849326518a611ee2/users/ef1449e1ec394177ba675fdc1c9bccbd/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:38.502583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9409356-7dee-4042-861d-87de2e0d35a2 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:38.502901 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9409356-7dee-4042-861d-87de2e0d35a2 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:38.503234 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9409356-7dee-4042-861d-87de2e0d35a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:38.541439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-d9409356-7dee-4042-861d-87de2e0d35a2 None None] MFA Rules not processed for user `ef1449e1ec394177ba675fdc1c9bccbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:38.542727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-d9409356-7dee-4042-861d-87de2e0d35a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:38.649357 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1158/2323] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:38.661993 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc93bc9-1ed7-4c6c-b994-873a64d910af admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:38.665211 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0fc93bc9-1ed7-4c6c-b994-873a64d910af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:38.766463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:38.789313 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc93bc9-1ed7-4c6c-b994-873a64d910af admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:38.793320 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1fd45-4952-433b-8ccd-53eaef7f6573 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:38.794168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1fd45-4952-433b-8ccd-53eaef7f6573 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:38.794572 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1fd45-4952-433b-8ccd-53eaef7f6573 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:38.795033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1fd45-4952-433b-8ccd-53eaef7f6573 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:38.808439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd1fd45-4952-433b-8ccd-53eaef7f6573 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:38.809941 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd1fd45-4952-433b-8ccd-53eaef7f6573 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:38.821109 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1166/2324] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:40:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:40:48.645231 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=51,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:48.815601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=8 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:51.928384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced196c6-7742-4428-b1c4-a6ef6d7f50c1 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:51.929425 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced196c6-7742-4428-b1c4-a6ef6d7f50c1 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:51.929515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced196c6-7742-4428-b1c4-a6ef6d7f50c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:51.947492 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-ced196c6-7742-4428-b1c4-a6ef6d7f50c1 None None] MFA Rules not processed for user `e9e358afe0974c0c8bd53104ebbff445`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:51.948841 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ced196c6-7742-4428-b1c4-a6ef6d7f50c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:52.012718 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1159/2325] 10.222.0.42 () {58 vars in 967 bytes} [Mon Jun 6 10:40:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:52.025996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1fd45-4952-433b-8ccd-53eaef7f6573 service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:52.027441 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bdd1fd45-4952-433b-8ccd-53eaef7f6573 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:52.103452 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1fd45-4952-433b-8ccd-53eaef7f6573 service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:52.108472 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3ae75-42ed-4c5a-886e-920504955a92 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fede9e52ad274285b09d748a684ede38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:52.109121 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3ae75-42ed-4c5a-886e-920504955a92 service swift] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:52.109402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3ae75-42ed-4c5a-886e-920504955a92 service swift] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:52.109675 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3ae75-42ed-4c5a-886e-920504955a92 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:52.118356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3ae75-42ed-4c5a-886e-920504955a92 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:52.120000 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3ae75-42ed-4c5a-886e-920504955a92 service swift] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:52.128317 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1167/2326] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jun 6 10:40:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 10:40:54.577972 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced196c6-7742-4428-b1c4-a6ef6d7f50c1 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:54.579408 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ced196c6-7742-4428-b1c4-a6ef6d7f50c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:54.651833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced196c6-7742-4428-b1c4-a6ef6d7f50c1 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:54.656768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f9cfc-e92d-49cd-ac8e-022b8b2a752b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:54.657645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f9cfc-e92d-49cd-ac8e-022b8b2a752b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:54.657929 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f9cfc-e92d-49cd-ac8e-022b8b2a752b admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:54.658199 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f9cfc-e92d-49cd-ac8e-022b8b2a752b admin admin] PATH_INFO: `/v3/users/717e1df58f404fa6b12b3f614c9425f4` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:54.659410 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f9cfc-e92d-49cd-ac8e-022b8b2a752b admin admin] RBAC: Authorizing `identity:delete_user(user_id=717e1df58f404fa6b12b3f614c9425f4)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:54.661404 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f9cfc-e92d-49cd-ac8e-022b8b2a752b admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:54.713853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2e1f9cfc-e92d-49cd-ac8e-022b8b2a752b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '717e1df58f404fa6b12b3f614c9425f4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:54.721426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2e1f9cfc-e92d-49cd-ac8e-022b8b2a752b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '717e1df58f404fa6b12b3f614c9425f4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:54.731997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2e1f9cfc-e92d-49cd-ac8e-022b8b2a752b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '717e1df58f404fa6b12b3f614c9425f4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:54.732701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2e1f9cfc-e92d-49cd-ac8e-022b8b2a752b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '717e1df58f404fa6b12b3f614c9425f4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:54.746756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2e1f9cfc-e92d-49cd-ac8e-022b8b2a752b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '717e1df58f404fa6b12b3f614c9425f4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:54.752636 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1160/2327] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:40:54 2022] DELETE /identity/v3/users/717e1df58f404fa6b12b3f614c9425f4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:54.757909 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648fda8c-ed22-49e4-8642-741d7fee0588 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:54.758397 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648fda8c-ed22-49e4-8642-741d7fee0588 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:54.758615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648fda8c-ed22-49e4-8642-741d7fee0588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:54.776463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-648fda8c-ed22-49e4-8642-741d7fee0588 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:54.777940 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-648fda8c-ed22-49e4-8642-741d7fee0588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:54.845869 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1168/2328] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:54.856090 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f9cfc-e92d-49cd-ac8e-022b8b2a752b admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:54.857561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-2e1f9cfc-e92d-49cd-ac8e-022b8b2a752b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:54.892373 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:54.946097 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f9cfc-e92d-49cd-ac8e-022b8b2a752b admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:54.950477 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc80bf-6655-4245-bece-7ffe6411db78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:54.951241 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc80bf-6655-4245-bece-7ffe6411db78 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:54.951527 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc80bf-6655-4245-bece-7ffe6411db78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:54.951954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc80bf-6655-4245-bece-7ffe6411db78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:54.964500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc80bf-6655-4245-bece-7ffe6411db78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:54.966243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc80bf-6655-4245-bece-7ffe6411db78 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:54.986805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1161/2329] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:40:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:40:55.371993 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606cd993-180a-475e-b62d-9e2c1f84d9d3 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:55.372217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606cd993-180a-475e-b62d-9e2c1f84d9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:55.372494 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606cd993-180a-475e-b62d-9e2c1f84d9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:55.405761 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-606cd993-180a-475e-b62d-9e2c1f84d9d3 None None] MFA Rules not processed for user `e9e358afe0974c0c8bd53104ebbff445`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:55.407012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-606cd993-180a-475e-b62d-9e2c1f84d9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:55.424086 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ffdee-f65d-4eb0-9be5-9465b030b526 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:55.424383 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ffdee-f65d-4eb0-9be5-9465b030b526 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:55.424666 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ffdee-f65d-4eb0-9be5-9465b030b526 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:55.426158 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1162/2330] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jun 6 10:40:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:40:55.436242 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ffdee-f65d-4eb0-9be5-9465b030b526 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:55.438041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-052ffdee-f65d-4eb0-9be5-9465b030b526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:55.475493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ffdee-f65d-4eb0-9be5-9465b030b526 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:55.493463 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab4b17-d701-4d25-9889-8c4054f3911f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:55.495280 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab4b17-d701-4d25-9889-8c4054f3911f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:55.495564 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab4b17-d701-4d25-9889-8c4054f3911f admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:55.496591 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab4b17-d701-4d25-9889-8c4054f3911f admin admin] PATH_INFO: `/v3/projects/b094932771b5414f90f863f844161030` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:55.497826 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab4b17-d701-4d25-9889-8c4054f3911f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b094932771b5414f90f863f844161030)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:55.499762 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab4b17-d701-4d25-9889-8c4054f3911f admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:55.542062 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1169/2331] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jun 6 10:40:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:55.544058 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13884f47-ebe3-4a6c-ad59-cc246bca877c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:55.544363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13884f47-ebe3-4a6c-ad59-cc246bca877c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:55.544641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13884f47-ebe3-4a6c-ad59-cc246bca877c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:55.591356 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-31ab4b17-d701-4d25-9889-8c4054f3911f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b094932771b5414f90f863f844161030', 'request_id': 'req-31ab4b17-d701-4d25-9889-8c4054f3911f'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:55.599806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-31ab4b17-d701-4d25-9889-8c4054f3911f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b094932771b5414f90f863f844161030', 'request_id': 'req-31ab4b17-d701-4d25-9889-8c4054f3911f'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:55.612441 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-31ab4b17-d701-4d25-9889-8c4054f3911f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b094932771b5414f90f863f844161030', 'request_id': 'req-31ab4b17-d701-4d25-9889-8c4054f3911f'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:55.629509 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-31ab4b17-d701-4d25-9889-8c4054f3911f admin admin] The token cache is being invalidate because project b094932771b5414f90f863f844161030 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:40:55.629810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-31ab4b17-d701-4d25-9889-8c4054f3911f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:55.630493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-31ab4b17-d701-4d25-9889-8c4054f3911f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:55.635614 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1163/2332] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:40:55 2022] DELETE /identity/v3/projects/b094932771b5414f90f863f844161030 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:55.635614 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab4b17-d701-4d25-9889-8c4054f3911f admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:55.635614 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-31ab4b17-d701-4d25-9889-8c4054f3911f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:55.640761 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-13884f47-ebe3-4a6c-ad59-cc246bca877c None None] MFA Rules not processed for user `361be69f1c0a41c18afd206bccf56447`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:55.642248 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-13884f47-ebe3-4a6c-ad59-cc246bca877c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:55.650085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:55.721530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab4b17-d701-4d25-9889-8c4054f3911f admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:55.727111 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b76d6dd-2f28-451f-a7cd-7ea831f43c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:55.728187 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76d6dd-2f28-451f-a7cd-7ea831f43c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:55.728499 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76d6dd-2f28-451f-a7cd-7ea831f43c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:55.728772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76d6dd-2f28-451f-a7cd-7ea831f43c2d admin admin] PATH_INFO: `/v3/users/446b13b4661542c097afafb32443dae4` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:55.730012 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b76d6dd-2f28-451f-a7cd-7ea831f43c2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=446b13b4661542c097afafb32443dae4)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:55.732042 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b76d6dd-2f28-451f-a7cd-7ea831f43c2d admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:55.746619 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1170/2333] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jun 6 10:40:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:55.748724 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13884f47-ebe3-4a6c-ad59-cc246bca877c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:55.750843 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-13884f47-ebe3-4a6c-ad59-cc246bca877c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:55.789758 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:55.791992 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1b76d6dd-2f28-451f-a7cd-7ea831f43c2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '446b13b4661542c097afafb32443dae4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:55.801991 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1b76d6dd-2f28-451f-a7cd-7ea831f43c2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '446b13b4661542c097afafb32443dae4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:55.810602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1b76d6dd-2f28-451f-a7cd-7ea831f43c2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '446b13b4661542c097afafb32443dae4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:55.811281 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1b76d6dd-2f28-451f-a7cd-7ea831f43c2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '446b13b4661542c097afafb32443dae4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:55.826137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1b76d6dd-2f28-451f-a7cd-7ea831f43c2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '446b13b4661542c097afafb32443dae4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:55.832535 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1164/2334] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:40:55 2022] DELETE /identity/v3/users/446b13b4661542c097afafb32443dae4 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:55.833832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619482cf-a64a-4a3f-b658-8b200432eb21 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:55.834175 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619482cf-a64a-4a3f-b658-8b200432eb21 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:55.835246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619482cf-a64a-4a3f-b658-8b200432eb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:55.845244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13884f47-ebe3-4a6c-ad59-cc246bca877c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:55.853400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-619482cf-a64a-4a3f-b658-8b200432eb21 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:55.854602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-619482cf-a64a-4a3f-b658-8b200432eb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:55.923544 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1165/2335] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:55.924567 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619482cf-a64a-4a3f-b658-8b200432eb21 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:55.925880 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-619482cf-a64a-4a3f-b658-8b200432eb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:55.932999 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd4262-db4c-497d-8002-702a059cc1d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fede9e52ad274285b09d748a684ede38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:55.934069 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd4262-db4c-497d-8002-702a059cc1d4 service swift] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:55.934421 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd4262-db4c-497d-8002-702a059cc1d4 service swift] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:55.934704 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd4262-db4c-497d-8002-702a059cc1d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:55.937041 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-98cd4262-db4c-497d-8002-702a059cc1d4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:55.990208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:56.010638 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd4262-db4c-497d-8002-702a059cc1d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:56.012307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd4262-db4c-497d-8002-702a059cc1d4 service swift] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:56.019981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619482cf-a64a-4a3f-b658-8b200432eb21 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:56.024706 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1171/2336] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jun 6 10:40:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 10:40:56.026402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3171f-8db6-400f-9f07-0234f28566dc None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:56.026787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3171f-8db6-400f-9f07-0234f28566dc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:56.026787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3171f-8db6-400f-9f07-0234f28566dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:56.033912 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8efd931-55d0-4926-824e-d418316d1718 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361be69f1c0a41c18afd206bccf56447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:56.035645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8efd931-55d0-4926-824e-d418316d1718 service placement] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:56.035975 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8efd931-55d0-4926-824e-d418316d1718 service placement] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:56.036269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8efd931-55d0-4926-824e-d418316d1718 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:56.039023 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-d8efd931-55d0-4926-824e-d418316d1718 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:56.049479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c8a3171f-8db6-400f-9f07-0234f28566dc None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:56.049479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c8a3171f-8db6-400f-9f07-0234f28566dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:56.054372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8efd931-55d0-4926-824e-d418316d1718 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:56.056114 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8efd931-55d0-4926-824e-d418316d1718 service placement] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:56.062807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1172/2337] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:56.062807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3171f-8db6-400f-9f07-0234f28566dc None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:56.070841 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1166/2338] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:40:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:40:56.072139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8efd931-55d0-4926-824e-d418316d1718 service placement] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:56.073007 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c8a3171f-8db6-400f-9f07-0234f28566dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:56.092173 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3171f-8db6-400f-9f07-0234f28566dc None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:56.092503 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8efd931-55d0-4926-824e-d418316d1718 service placement] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:56.103768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85dfa33-4962-4674-ae0f-0b6fb38199b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:56.104904 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85dfa33-4962-4674-ae0f-0b6fb38199b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:56.105338 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85dfa33-4962-4674-ae0f-0b6fb38199b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:56.105664 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85dfa33-4962-4674-ae0f-0b6fb38199b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:56.107163 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85dfa33-4962-4674-ae0f-0b6fb38199b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:40:56.107573 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85dfa33-4962-4674-ae0f-0b6fb38199b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:56.109565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85dfa33-4962-4674-ae0f-0b6fb38199b5 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:56.114093 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c55afa-96ea-4907-a032-d088c8acb9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:56.116988 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c55afa-96ea-4907-a032-d088c8acb9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:56.117854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c55afa-96ea-4907-a032-d088c8acb9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:56.118306 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c55afa-96ea-4907-a032-d088c8acb9a8 admin admin] PATH_INFO: `/v3/projects/b3b18ee23fb548709fc513c7e1cfe623` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:56.121975 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c55afa-96ea-4907-a032-d088c8acb9a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3b18ee23fb548709fc513c7e1cfe623)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:56.126189 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c55afa-96ea-4907-a032-d088c8acb9a8 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:56.133594 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1167/2339] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:40:56.138177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85dfa33-4962-4674-ae0f-0b6fb38199b5 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:56.150745 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85dfa33-4962-4674-ae0f-0b6fb38199b5 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:56.156624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636b1f7-daa6-45c8-b6d5-f37bca01c51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:56.157283 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636b1f7-daa6-45c8-b6d5-f37bca01c51d admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:56.157562 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636b1f7-daa6-45c8-b6d5-f37bca01c51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:56.157832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636b1f7-daa6-45c8-b6d5-f37bca01c51d admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:56.158601 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d636b1f7-daa6-45c8-b6d5-f37bca01c51d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:56.161559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d636b1f7-daa6-45c8-b6d5-f37bca01c51d admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:56.196145 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1168/2340] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:40:56 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:40:56.199054 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636b1f7-daa6-45c8-b6d5-f37bca01c51d admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:56.222710 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d3c55afa-96ea-4907-a032-d088c8acb9a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3b18ee23fb548709fc513c7e1cfe623', 'request_id': 'req-d3c55afa-96ea-4907-a032-d088c8acb9a8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:56.226785 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:40:56.235693 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d3c55afa-96ea-4907-a032-d088c8acb9a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3b18ee23fb548709fc513c7e1cfe623', 'request_id': 'req-d3c55afa-96ea-4907-a032-d088c8acb9a8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:56.245657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d3c55afa-96ea-4907-a032-d088c8acb9a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3b18ee23fb548709fc513c7e1cfe623', 'request_id': 'req-d3c55afa-96ea-4907-a032-d088c8acb9a8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:56.263806 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d3c55afa-96ea-4907-a032-d088c8acb9a8 admin admin] The token cache is being invalidate because project b3b18ee23fb548709fc513c7e1cfe623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:40:56.264129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d3c55afa-96ea-4907-a032-d088c8acb9a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:56.265097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d3c55afa-96ea-4907-a032-d088c8acb9a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:40:56.266973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1173/2341] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:40:56 2022] DELETE /identity/v3/projects/b3b18ee23fb548709fc513c7e1cfe623 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:40:56.307048 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636b1f7-daa6-45c8-b6d5-f37bca01c51d admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:56.389646 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f518007-1850-452b-b299-7b1e2e301dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:56.390423 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f518007-1850-452b-b299-7b1e2e301dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:56.391055 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f518007-1850-452b-b299-7b1e2e301dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:56.391467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f518007-1850-452b-b299-7b1e2e301dd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:56.392459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f518007-1850-452b-b299-7b1e2e301dd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:56.394846 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f518007-1850-452b-b299-7b1e2e301dd5 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:56.434666 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1169/2342] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:40:56 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:40:56.441500 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c55afa-96ea-4907-a032-d088c8acb9a8 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:56.451818 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d3c55afa-96ea-4907-a032-d088c8acb9a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:56.491829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c55afa-96ea-4907-a032-d088c8acb9a8 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:56.501982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4bc185-6b5c-402e-9cec-6512ee4d1b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:56.502585 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4bc185-6b5c-402e-9cec-6512ee4d1b88 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:56.502866 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4bc185-6b5c-402e-9cec-6512ee4d1b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:56.503143 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4bc185-6b5c-402e-9cec-6512ee4d1b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:56.504552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4bc185-6b5c-402e-9cec-6512ee4d1b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:56.508471 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4bc185-6b5c-402e-9cec-6512ee4d1b88 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:56.520787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1174/2343] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:40:56.525480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f518007-1850-452b-b299-7b1e2e301dd5 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:56.538983 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f518007-1850-452b-b299-7b1e2e301dd5 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:56.544710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916aa28-b7e1-490b-a19c-e135e7652b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:56.545390 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916aa28-b7e1-490b-a19c-e135e7652b88 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:56.545670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916aa28-b7e1-490b-a19c-e135e7652b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:56.545950 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916aa28-b7e1-490b-a19c-e135e7652b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:56.546710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916aa28-b7e1-490b-a19c-e135e7652b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:56.548358 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916aa28-b7e1-490b-a19c-e135e7652b88 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:56.563217 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1170/2344] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:40:56.567806 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4bc185-6b5c-402e-9cec-6512ee4d1b88 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:56.579020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4bc185-6b5c-402e-9cec-6512ee4d1b88 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:56.583669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb74760-6b5b-4b58-b653-4ba5563db95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:56.584647 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb74760-6b5b-4b58-b653-4ba5563db95d admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:56.584931 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb74760-6b5b-4b58-b653-4ba5563db95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:56.585388 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb74760-6b5b-4b58-b653-4ba5563db95d admin admin] PATH_INFO: `/v3/projects/645eadf589944a24bf6e27326a61952c/users/b7b11ed004fc4ef3a1f1c48adf31fe8e/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:56.603697 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb74760-6b5b-4b58-b653-4ba5563db95d admin admin] RBAC: Authorizing `identity:create_grant(project_id=645eadf589944a24bf6e27326a61952c, user_id=b7b11ed004fc4ef3a1f1c48adf31fe8e, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:56.606892 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb74760-6b5b-4b58-b653-4ba5563db95d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:56.624778 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1175/2345] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:40:56 2022] PUT /identity/v3/projects/645eadf589944a24bf6e27326a61952c/users/b7b11ed004fc4ef3a1f1c48adf31fe8e/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:40:56.630892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87729225-ce57-4b26-9f67-590a7519bd83 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:56.631461 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87729225-ce57-4b26-9f67-590a7519bd83 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:56.631745 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87729225-ce57-4b26-9f67-590a7519bd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:56.650839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-87729225-ce57-4b26-9f67-590a7519bd83 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:40:56.652209 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-87729225-ce57-4b26-9f67-590a7519bd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:56.735624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1171/2346] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:40:56.745552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb74760-6b5b-4b58-b653-4ba5563db95d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:40:56.747388 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ddb74760-6b5b-4b58-b653-4ba5563db95d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:40:56.812621 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb74760-6b5b-4b58-b653-4ba5563db95d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:40:56.820275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0bd4bb-c3b4-4da7-a2fe-7fe0927dea8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:40:56.821003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0bd4bb-c3b4-4da7-a2fe-7fe0927dea8c service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:40:56.821290 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0bd4bb-c3b4-4da7-a2fe-7fe0927dea8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:40:56.821563 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0bd4bb-c3b4-4da7-a2fe-7fe0927dea8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:40:56.834949 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0bd4bb-c3b4-4da7-a2fe-7fe0927dea8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:40:56.838333 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0bd4bb-c3b4-4da7-a2fe-7fe0927dea8c service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:40:56.855412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1176/2347] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:40:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:41:00.355677 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87729225-ce57-4b26-9f67-590a7519bd83 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:41:00.357669 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-87729225-ce57-4b26-9f67-590a7519bd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:00.375856 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87729225-ce57-4b26-9f67-590a7519bd83 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:41:00.390708 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04044cb9-9ece-42da-9d13-6d227f7f4e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:41:00.391655 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04044cb9-9ece-42da-9d13-6d227f7f4e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:00.392180 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04044cb9-9ece-42da-9d13-6d227f7f4e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:00.392593 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04044cb9-9ece-42da-9d13-6d227f7f4e54 admin admin] PATH_INFO: `/v3/users/ef1449e1ec394177ba675fdc1c9bccbd` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:00.395539 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04044cb9-9ece-42da-9d13-6d227f7f4e54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef1449e1ec394177ba675fdc1c9bccbd)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:41:00.400338 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04044cb9-9ece-42da-9d13-6d227f7f4e54 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:41:00.509294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:41:00.522986 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-04044cb9-9ece-42da-9d13-6d227f7f4e54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef1449e1ec394177ba675fdc1c9bccbd'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:41:00.543564 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-04044cb9-9ece-42da-9d13-6d227f7f4e54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef1449e1ec394177ba675fdc1c9bccbd'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:41:00.581890 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-04044cb9-9ece-42da-9d13-6d227f7f4e54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef1449e1ec394177ba675fdc1c9bccbd'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:41:00.583468 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-04044cb9-9ece-42da-9d13-6d227f7f4e54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef1449e1ec394177ba675fdc1c9bccbd'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:41:00.613531 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-04044cb9-9ece-42da-9d13-6d227f7f4e54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef1449e1ec394177ba675fdc1c9bccbd'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:41:00.635638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1172/2348] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:41:00 2022] DELETE /identity/v3/users/ef1449e1ec394177ba675fdc1c9bccbd => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:41:00.639912 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65937474-c882-45c4-b20a-25c6bebad761 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:00.640401 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65937474-c882-45c4-b20a-25c6bebad761 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:00.640524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65937474-c882-45c4-b20a-25c6bebad761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:00.691659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-65937474-c882-45c4-b20a-25c6bebad761 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:41:00.703441 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-65937474-c882-45c4-b20a-25c6bebad761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:00.793173 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:41:00.876106 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1177/2349] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:41:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:41:00.888872 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04044cb9-9ece-42da-9d13-6d227f7f4e54 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:41:00.892697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-04044cb9-9ece-42da-9d13-6d227f7f4e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:01.061566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04044cb9-9ece-42da-9d13-6d227f7f4e54 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:41:01.066247 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9d451-aabd-4fcd-b37e-6c396c7250bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:41:01.067096 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9d451-aabd-4fcd-b37e-6c396c7250bb service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:01.067486 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9d451-aabd-4fcd-b37e-6c396c7250bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:01.067940 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9d451-aabd-4fcd-b37e-6c396c7250bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:01.085051 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9d451-aabd-4fcd-b37e-6c396c7250bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:41:01.087076 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9d451-aabd-4fcd-b37e-6c396c7250bb service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:41:01.115057 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1173/2350] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:41:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:41:01.688184 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65937474-c882-45c4-b20a-25c6bebad761 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:41:01.691808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-65937474-c882-45c4-b20a-25c6bebad761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:01.715587 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65937474-c882-45c4-b20a-25c6bebad761 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:41:01.743472 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda0179-5505-4f27-a23b-18c312ab7581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:41:01.744412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda0179-5505-4f27-a23b-18c312ab7581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:01.744695 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda0179-5505-4f27-a23b-18c312ab7581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:01.744968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda0179-5505-4f27-a23b-18c312ab7581 admin admin] PATH_INFO: `/v3/projects/4e6700296b224ffa849326518a611ee2` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:01.750563 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda0179-5505-4f27-a23b-18c312ab7581 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e6700296b224ffa849326518a611ee2)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:41:01.753716 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda0179-5505-4f27-a23b-18c312ab7581 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:41:01.871115 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0fda0179-5505-4f27-a23b-18c312ab7581 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e6700296b224ffa849326518a611ee2', 'request_id': 'req-0fda0179-5505-4f27-a23b-18c312ab7581'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:41:01.891003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0fda0179-5505-4f27-a23b-18c312ab7581 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e6700296b224ffa849326518a611ee2', 'request_id': 'req-0fda0179-5505-4f27-a23b-18c312ab7581'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:41:01.900130 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0fda0179-5505-4f27-a23b-18c312ab7581 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e6700296b224ffa849326518a611ee2', 'request_id': 'req-0fda0179-5505-4f27-a23b-18c312ab7581'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:41:01.936531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0fda0179-5505-4f27-a23b-18c312ab7581 admin admin] The token cache is being invalidate because project 4e6700296b224ffa849326518a611ee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:41:01.937235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0fda0179-5505-4f27-a23b-18c312ab7581 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:41:01.938745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0fda0179-5505-4f27-a23b-18c312ab7581 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:41:01.956300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1178/2351] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:41:01 2022] DELETE /identity/v3/projects/4e6700296b224ffa849326518a611ee2 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:41:03.786398 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7a551-adfc-42e7-ba0a-b39f2d01273a None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:03.786876 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7a551-adfc-42e7-ba0a-b39f2d01273a None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:03.786980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7a551-adfc-42e7-ba0a-b39f2d01273a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:03.863123 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-f8b7a551-adfc-42e7-ba0a-b39f2d01273a None None] MFA Rules not processed for user `b7b11ed004fc4ef3a1f1c48adf31fe8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:41:03.864406 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-f8b7a551-adfc-42e7-ba0a-b39f2d01273a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:03.913306 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda0179-5505-4f27-a23b-18c312ab7581 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:41:03.915735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0fda0179-5505-4f27-a23b-18c312ab7581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:04.003838 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1174/2352] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:41:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:41:04.004749 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda0179-5505-4f27-a23b-18c312ab7581 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:41:04.006805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7a551-adfc-42e7-ba0a-b39f2d01273a None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:41:04.009929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28125e-7f6f-463e-947c-fb2d9f640775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:41:04.010600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28125e-7f6f-463e-947c-fb2d9f640775 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:04.010887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28125e-7f6f-463e-947c-fb2d9f640775 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:04.011165 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28125e-7f6f-463e-947c-fb2d9f640775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:04.011636 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-f8b7a551-adfc-42e7-ba0a-b39f2d01273a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:04.013349 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6c28125e-7f6f-463e-947c-fb2d9f640775 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:04.102260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28125e-7f6f-463e-947c-fb2d9f640775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:41:04.103744 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28125e-7f6f-463e-947c-fb2d9f640775 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:41:04.107578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:41:04.110454 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:41:04.117363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1179/2353] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:41:03 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:41:04.148626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7a551-adfc-42e7-ba0a-b39f2d01273a None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:41:04.153753 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1028f5a-7ecc-4cb2-9e45-dff55ff40493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:41:04.154760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1028f5a-7ecc-4cb2-9e45-dff55ff40493 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:04.154760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1028f5a-7ecc-4cb2-9e45-dff55ff40493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:04.155072 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1028f5a-7ecc-4cb2-9e45-dff55ff40493 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:04.155606 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1028f5a-7ecc-4cb2-9e45-dff55ff40493 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:41:04.157712 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1028f5a-7ecc-4cb2-9e45-dff55ff40493 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:41:04.193304 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1175/2354] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:41:04 2022] POST /identity/v3/projects => generated 383 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:41:04.197113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28125e-7f6f-463e-947c-fb2d9f640775 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:41:04.213848 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28125e-7f6f-463e-947c-fb2d9f640775 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:41:04.291825 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e60f6-01a3-4e82-af2d-955316ababe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:41:04.292460 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e60f6-01a3-4e82-af2d-955316ababe6 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:04.292732 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e60f6-01a3-4e82-af2d-955316ababe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:04.293238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e60f6-01a3-4e82-af2d-955316ababe6 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:04.294150 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e60f6-01a3-4e82-af2d-955316ababe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:41:04.296079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e60f6-01a3-4e82-af2d-955316ababe6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:41:04.335627 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1180/2355] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:41:04 2022] POST /identity/v3/users => generated 298 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:41:04.339414 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1028f5a-7ecc-4cb2-9e45-dff55ff40493 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:41:04.351546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1028f5a-7ecc-4cb2-9e45-dff55ff40493 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:41:04.365109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c561c-45c3-4fb8-9db1-58d2d171c2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:41:04.366424 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c561c-45c3-4fb8-9db1-58d2d171c2ea admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:04.366577 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c561c-45c3-4fb8-9db1-58d2d171c2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:04.366861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c561c-45c3-4fb8-9db1-58d2d171c2ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:04.367596 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c561c-45c3-4fb8-9db1-58d2d171c2ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:41:04.369256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c561c-45c3-4fb8-9db1-58d2d171c2ea admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:41:04.382963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1176/2356] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:41:04.387055 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e60f6-01a3-4e82-af2d-955316ababe6 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:41:04.402352 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e60f6-01a3-4e82-af2d-955316ababe6 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:41:04.417409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f7f7f-c54e-4f7c-8e42-6a3fecf71e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:41:04.419399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f7f7f-c54e-4f7c-8e42-6a3fecf71e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:04.421634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f7f7f-c54e-4f7c-8e42-6a3fecf71e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:04.422218 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f7f7f-c54e-4f7c-8e42-6a3fecf71e74 admin admin] PATH_INFO: `/v3/projects/2dbd010f8c024d7dac26979caa814c4e/users/f791934803044943b030f1f1677e2361/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:04.460356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f7f7f-c54e-4f7c-8e42-6a3fecf71e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dbd010f8c024d7dac26979caa814c4e, user_id=f791934803044943b030f1f1677e2361, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:41:04.463425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f7f7f-c54e-4f7c-8e42-6a3fecf71e74 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:41:04.487825 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1181/2357] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:41:04 2022] PUT /identity/v3/projects/2dbd010f8c024d7dac26979caa814c4e/users/f791934803044943b030f1f1677e2361/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:41:04.490379 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c561c-45c3-4fb8-9db1-58d2d171c2ea admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:41:04.510379 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c561c-45c3-4fb8-9db1-58d2d171c2ea admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:41:04.631723 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e8dfc-437e-467a-9f0f-f5ee1d1975f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:41:04.632612 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e8dfc-437e-467a-9f0f-f5ee1d1975f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:04.633006 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e8dfc-437e-467a-9f0f-f5ee1d1975f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:04.633385 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e8dfc-437e-467a-9f0f-f5ee1d1975f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:04.634385 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e8dfc-437e-467a-9f0f-f5ee1d1975f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:41:04.640393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e8dfc-437e-467a-9f0f-f5ee1d1975f6 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:41:04.659679 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1177/2358] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:41:04.664058 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f7f7f-c54e-4f7c-8e42-6a3fecf71e74 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:41:04.677439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f7f7f-c54e-4f7c-8e42-6a3fecf71e74 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:41:04.692106 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc9952c-3adf-45f1-ab4e-2280ce0d9296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:41:04.693206 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9952c-3adf-45f1-ab4e-2280ce0d9296 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:04.695942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9952c-3adf-45f1-ab4e-2280ce0d9296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:04.695942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9952c-3adf-45f1-ab4e-2280ce0d9296 admin admin] PATH_INFO: `/v3/projects/2dbd010f8c024d7dac26979caa814c4e/users/f791934803044943b030f1f1677e2361/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:04.698475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc9952c-3adf-45f1-ab4e-2280ce0d9296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dbd010f8c024d7dac26979caa814c4e, user_id=f791934803044943b030f1f1677e2361, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:41:04.703425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc9952c-3adf-45f1-ab4e-2280ce0d9296 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:41:04.729425 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1182/2359] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:41:04 2022] PUT /identity/v3/projects/2dbd010f8c024d7dac26979caa814c4e/users/f791934803044943b030f1f1677e2361/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:41:05.762627 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e8dfc-437e-467a-9f0f-f5ee1d1975f6 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:41:05.765789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-c39e8dfc-437e-467a-9f0f-f5ee1d1975f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:05.925843 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e8dfc-437e-467a-9f0f-f5ee1d1975f6 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:41:06.044233 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:41:06.059244 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2efbd1-860b-4b04-9b33-b580a4911d13 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361be69f1c0a41c18afd206bccf56447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:41:06.060463 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2efbd1-860b-4b04-9b33-b580a4911d13 service placement] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:06.060752 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2efbd1-860b-4b04-9b33-b580a4911d13 service placement] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:06.061032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2efbd1-860b-4b04-9b33-b580a4911d13 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:06.063443 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-db2efbd1-860b-4b04-9b33-b580a4911d13 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:06.081889 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2efbd1-860b-4b04-9b33-b580a4911d13 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:41:06.084912 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2efbd1-860b-4b04-9b33-b580a4911d13 service placement] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:41:06.114593 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1178/2360] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:41:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:41:12.929630 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e695de4-fcae-49bb-91df-5b318abae2d8 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:12.930156 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e695de4-fcae-49bb-91df-5b318abae2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:12.930252 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e695de4-fcae-49bb-91df-5b318abae2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:12.962663 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2e695de4-fcae-49bb-91df-5b318abae2d8 None None] MFA Rules not processed for user `f791934803044943b030f1f1677e2361`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:41:12.964488 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2e695de4-fcae-49bb-91df-5b318abae2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:13.043013 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1183/2361] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:41:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:41:13.056270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2efbd1-860b-4b04-9b33-b580a4911d13 service placement] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:41:13.057925 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-db2efbd1-860b-4b04-9b33-b580a4911d13 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:13.156457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2efbd1-860b-4b04-9b33-b580a4911d13 service placement] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:41:13.160145 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34afdb25-2216-4d57-8522-3e5583afbfff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a67e007fc314b34aac2471a7c20f240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:41:13.160952 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afdb25-2216-4d57-8522-3e5583afbfff service glance] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:13.161242 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afdb25-2216-4d57-8522-3e5583afbfff service glance] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:13.161553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afdb25-2216-4d57-8522-3e5583afbfff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:13.229471 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34afdb25-2216-4d57-8522-3e5583afbfff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:41:13.231096 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34afdb25-2216-4d57-8522-3e5583afbfff service glance] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:41:13.241347 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1179/2362] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:41:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:41:13.367047 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ef02c-af9b-4d8e-8c24-695b036a9906 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:13.367436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ef02c-af9b-4d8e-8c24-695b036a9906 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:13.367719 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ef02c-af9b-4d8e-8c24-695b036a9906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:13.422443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-fa2ef02c-af9b-4d8e-8c24-695b036a9906 None None] MFA Rules not processed for user `e9e358afe0974c0c8bd53104ebbff445`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:41:13.423668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fa2ef02c-af9b-4d8e-8c24-695b036a9906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:13.434529 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:41:13.496926 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1184/2363] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jun 6 10:41:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:41:13.510241 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34afdb25-2216-4d57-8522-3e5583afbfff service glance] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:41:13.511526 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-34afdb25-2216-4d57-8522-3e5583afbfff service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:13.605833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34afdb25-2216-4d57-8522-3e5583afbfff service glance] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:41:13.610943 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd656d-e1bf-41ea-8540-cc0d22a86e24 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fede9e52ad274285b09d748a684ede38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:41:13.611589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd656d-e1bf-41ea-8540-cc0d22a86e24 service swift] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:13.611909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd656d-e1bf-41ea-8540-cc0d22a86e24 service swift] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:13.612192 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd656d-e1bf-41ea-8540-cc0d22a86e24 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:13.619382 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:41:13.621973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bd656d-e1bf-41ea-8540-cc0d22a86e24 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:41:13.624194 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bd656d-e1bf-41ea-8540-cc0d22a86e24 service swift] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:41:13.633756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1180/2364] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jun 6 10:41:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 10:41:17.503962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4f992-3389-42da-b4a9-544057bf0b42 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:17.504580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4f992-3389-42da-b4a9-544057bf0b42 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:17.504580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4f992-3389-42da-b4a9-544057bf0b42 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:17.506440 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1185/2365] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jun 6 10:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:41:17.514570 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ccc78-4f43-4fa0-ba81-0e5398abf8a5 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:17.515135 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ccc78-4f43-4fa0-ba81-0e5398abf8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:17.515308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ccc78-4f43-4fa0-ba81-0e5398abf8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:17.639877 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-033ccc78-4f43-4fa0-ba81-0e5398abf8a5 None None] MFA Rules not processed for user `ed8a40e4f1034a8e99d0eb505e448db9`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:41:17.641262 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-033ccc78-4f43-4fa0-ba81-0e5398abf8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:17.783109 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1181/2366] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jun 6 10:41:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:41:17.799563 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4f992-3389-42da-b4a9-544057bf0b42 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:41:17.811064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4f992-3389-42da-b4a9-544057bf0b42 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:41:17.815336 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6369e-82dc-41a5-89f3-0658d5c51342 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a67e007fc314b34aac2471a7c20f240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:41:17.815520 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6369e-82dc-41a5-89f3-0658d5c51342 service glance] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:17.815702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6369e-82dc-41a5-89f3-0658d5c51342 service glance] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:17.816366 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6369e-82dc-41a5-89f3-0658d5c51342 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:17.831147 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6369e-82dc-41a5-89f3-0658d5c51342 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:41:17.835423 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6369e-82dc-41a5-89f3-0658d5c51342 service glance] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:41:17.847548 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1186/2367] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:41:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:41:17.923590 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a5a0d-1e31-491a-842b-cd40a4144b29 None None] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:17.924068 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a5a0d-1e31-491a-842b-cd40a4144b29 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:17.924397 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a5a0d-1e31-491a-842b-cd40a4144b29 None None] PATH_INFO: `/` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:17.925868 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1182/2368] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jun 6 10:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:41:17.931564 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad454e6-959f-41a6-ac15-b5b513fd2080 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:17.931950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad454e6-959f-41a6-ac15-b5b513fd2080 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:17.932241 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad454e6-959f-41a6-ac15-b5b513fd2080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:17.951993 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3ad454e6-959f-41a6-ac15-b5b513fd2080 None None] MFA Rules not processed for user `881069f9b40a485dac2dda684dd6cd3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:41:17.953484 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3ad454e6-959f-41a6-ac15-b5b513fd2080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:18.019326 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1187/2369] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jun 6 10:41:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:41:18.033510 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a5a0d-1e31-491a-842b-cd40a4144b29 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:41:18.035403 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-0e7a5a0d-1e31-491a-842b-cd40a4144b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:18.053347 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a5a0d-1e31-491a-842b-cd40a4144b29 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:41:18.057703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf488948-709a-4de7-857e-ce6a26d5a17a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:41:18.058551 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf488948-709a-4de7-857e-ce6a26d5a17a service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:18.058836 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf488948-709a-4de7-857e-ce6a26d5a17a service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:18.059136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf488948-709a-4de7-857e-ce6a26d5a17a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:18.067275 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf488948-709a-4de7-857e-ce6a26d5a17a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:41:18.069246 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf488948-709a-4de7-857e-ce6a26d5a17a service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:41:18.078560 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1183/2370] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:41:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:41:18.181014 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf2d5b-2427-47a5-a20f-0770dfb44fc6 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:18.181307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf2d5b-2427-47a5-a20f-0770dfb44fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:18.181587 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf2d5b-2427-47a5-a20f-0770dfb44fc6 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:18.183003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1188/2371] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jun 6 10:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 10:41:18.191217 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165d463-c2ba-4b13-a20e-0cfd6349ac6f None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:18.191217 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165d463-c2ba-4b13-a20e-0cfd6349ac6f None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:18.193118 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165d463-c2ba-4b13-a20e-0cfd6349ac6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:18.213958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-0165d463-c2ba-4b13-a20e-0cfd6349ac6f None None] MFA Rules not processed for user `ed8a40e4f1034a8e99d0eb505e448db9`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:41:18.215758 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-0165d463-c2ba-4b13-a20e-0cfd6349ac6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:18.226264 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1184/2372] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jun 6 10:41:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:41:18.239874 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf2d5b-2427-47a5-a20f-0770dfb44fc6 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:41:18.251342 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf2d5b-2427-47a5-a20f-0770dfb44fc6 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:41:18.257253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2ba44-eb65-48fa-8b1d-712c7d92b9df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:41:18.257253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2ba44-eb65-48fa-8b1d-712c7d92b9df service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:18.257253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2ba44-eb65-48fa-8b1d-712c7d92b9df service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:18.257253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2ba44-eb65-48fa-8b1d-712c7d92b9df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:18.276752 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b2ba44-eb65-48fa-8b1d-712c7d92b9df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:41:18.278381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b2ba44-eb65-48fa-8b1d-712c7d92b9df service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:41:18.290541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1189/2373] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:41:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:41:20.846945 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0dfde-6d55-4dad-9038-0b9f1dc35dd9 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:20.847488 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0dfde-6d55-4dad-9038-0b9f1dc35dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:20.847665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0dfde-6d55-4dad-9038-0b9f1dc35dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:20.869441 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-90c0dfde-6d55-4dad-9038-0b9f1dc35dd9 None None] MFA Rules not processed for user `e9e358afe0974c0c8bd53104ebbff445`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:41:20.870758 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-90c0dfde-6d55-4dad-9038-0b9f1dc35dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:20.881997 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1185/2374] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jun 6 10:41:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:41:20.900635 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2ba44-eb65-48fa-8b1d-712c7d92b9df service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:41:20.913880 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2ba44-eb65-48fa-8b1d-712c7d92b9df service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:41:20.922086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc8b030-cf4c-4b90-8f87-5ef14d928bed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fede9e52ad274285b09d748a684ede38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:41:20.922741 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8b030-cf4c-4b90-8f87-5ef14d928bed service swift] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:20.923023 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8b030-cf4c-4b90-8f87-5ef14d928bed service swift] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:20.923294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8b030-cf4c-4b90-8f87-5ef14d928bed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:20.933646 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc8b030-cf4c-4b90-8f87-5ef14d928bed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:41:20.935076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc8b030-cf4c-4b90-8f87-5ef14d928bed service swift] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:41:20.944590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1190/2375] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jun 6 10:41:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 10:41:23.630367 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652c116-91e8-4747-87b3-be26c7f0acbf None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:23.630976 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652c116-91e8-4747-87b3-be26c7f0acbf None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:23.630976 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652c116-91e8-4747-87b3-be26c7f0acbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:23.651632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-0652c116-91e8-4747-87b3-be26c7f0acbf None None] MFA Rules not processed for user `b7b11ed004fc4ef3a1f1c48adf31fe8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:41:23.652901 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-0652c116-91e8-4747-87b3-be26c7f0acbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:23.659938 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1186/2376] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:41:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:41:30.940792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=64 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:41:31.936326 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174aa4eb-3c72-4567-8911-6b812ff23136 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:31.936713 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174aa4eb-3c72-4567-8911-6b812ff23136 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:31.937204 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174aa4eb-3c72-4567-8911-6b812ff23136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:31.958194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-174aa4eb-3c72-4567-8911-6b812ff23136 None None] MFA Rules not processed for user `e9e358afe0974c0c8bd53104ebbff445`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:41:31.959904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-174aa4eb-3c72-4567-8911-6b812ff23136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:31.968070 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1191/2377] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jun 6 10:41:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:41:31.991517 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652c116-91e8-4747-87b3-be26c7f0acbf None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:41:32.003514 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652c116-91e8-4747-87b3-be26c7f0acbf None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:41:32.009116 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a81f0-61c1-4ebd-8586-0fece146cea4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fede9e52ad274285b09d748a684ede38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:41:32.009752 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a81f0-61c1-4ebd-8586-0fece146cea4 service swift] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:32.010030 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a81f0-61c1-4ebd-8586-0fece146cea4 service swift] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:32.010299 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a81f0-61c1-4ebd-8586-0fece146cea4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:32.018723 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a81f0-61c1-4ebd-8586-0fece146cea4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:41:32.020389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a81f0-61c1-4ebd-8586-0fece146cea4 service swift] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:41:32.029231 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1187/2378] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jun 6 10:41:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 10:41:41.951587 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:41:42.025761 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=69 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:41:47.997615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd51e9c-c565-45d2-86e0-fe1fff4959ce None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:47.998089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd51e9c-c565-45d2-86e0-fe1fff4959ce None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:47.998180 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd51e9c-c565-45d2-86e0-fe1fff4959ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:48.016636 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9cd51e9c-c565-45d2-86e0-fe1fff4959ce None None] MFA Rules not processed for user `e9e358afe0974c0c8bd53104ebbff445`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:41:48.018094 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9cd51e9c-c565-45d2-86e0-fe1fff4959ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:41:48.028262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1192/2379] 10.222.0.42 () {58 vars in 967 bytes} [Mon Jun 6 10:41:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:41:48.040959 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a81f0-61c1-4ebd-8586-0fece146cea4 service swift] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:41:48.053089 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a81f0-61c1-4ebd-8586-0fece146cea4 service swift] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:41:48.058800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ce998-27ab-4b2f-82fe-6d7c3f6bb6dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fede9e52ad274285b09d748a684ede38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:41:48.059734 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ce998-27ab-4b2f-82fe-6d7c3f6bb6dd service swift] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:41:48.060275 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ce998-27ab-4b2f-82fe-6d7c3f6bb6dd service swift] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:41:48.060612 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ce998-27ab-4b2f-82fe-6d7c3f6bb6dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:41:48.071493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ce998-27ab-4b2f-82fe-6d7c3f6bb6dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:41:48.073400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ce998-27ab-4b2f-82fe-6d7c3f6bb6dd service swift] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:41:48.084229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1188/2380] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jun 6 10:41:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 10:41:58.011458 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:41:58.080171 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=6 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:42:01.962483 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd51e9c-c565-45d2-86e0-fe1fff4959ce None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:42:01.972555 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd51e9c-c565-45d2-86e0-fe1fff4959ce None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:42:01.976116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7096f-d115-413e-b213-65caea54465a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:42:01.976745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7096f-d115-413e-b213-65caea54465a service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:42:01.977024 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7096f-d115-413e-b213-65caea54465a service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:42:01.977293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7096f-d115-413e-b213-65caea54465a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:42:01.986079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7096f-d115-413e-b213-65caea54465a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:42:01.987813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7096f-d115-413e-b213-65caea54465a service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:42:01.996965 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1193/2381] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:42:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:42:11.992904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=6 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:42:14.638876 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691154d-e0c4-4e4d-b240-110db7693d31 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:42:14.639378 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691154d-e0c4-4e4d-b240-110db7693d31 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:42:14.639477 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691154d-e0c4-4e4d-b240-110db7693d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:42:14.658763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-8691154d-e0c4-4e4d-b240-110db7693d31 None None] MFA Rules not processed for user `f269e146c844480c822347c3a38d103b`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:42:14.660200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-8691154d-e0c4-4e4d-b240-110db7693d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:42:14.718850 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1189/2382] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:42:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:42:24.711620 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=25 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:42:37.663140 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a50034-119c-4953-ba0d-0b94f3920200 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:42:37.663621 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a50034-119c-4953-ba0d-0b94f3920200 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:42:37.663714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a50034-119c-4953-ba0d-0b94f3920200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:42:37.681212 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-10a50034-119c-4953-ba0d-0b94f3920200 None None] MFA Rules not processed for user `ea1482ba53554f71865cea46ffbac23c`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:42:37.682395 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-10a50034-119c-4953-ba0d-0b94f3920200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:42:37.734710 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1194/2383] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:42:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:42:44.210529 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7937e-d9cd-49c6-9b40-5ed6537af761 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:42:44.211063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7937e-d9cd-49c6-9b40-5ed6537af761 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:42:44.211163 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7937e-d9cd-49c6-9b40-5ed6537af761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:42:44.228823 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-69f7937e-d9cd-49c6-9b40-5ed6537af761 None None] MFA Rules not processed for user `f6b7d82c999646b6b6b96d565f0e3f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:42:44.230055 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-69f7937e-d9cd-49c6-9b40-5ed6537af761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:42:44.284793 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1190/2384] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:42:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:42:47.727899 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=25 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:42:54.277623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=25 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:43:04.154401 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a50034-119c-4953-ba0d-0b94f3920200 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:04.165020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a50034-119c-4953-ba0d-0b94f3920200 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:04.168773 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251d751-7c15-4803-8d79-640bcc426bfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:04.169494 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251d751-7c15-4803-8d79-640bcc426bfd service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:04.169825 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251d751-7c15-4803-8d79-640bcc426bfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:04.170109 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251d751-7c15-4803-8d79-640bcc426bfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:04.172114 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6251d751-7c15-4803-8d79-640bcc426bfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:04.185816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251d751-7c15-4803-8d79-640bcc426bfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:04.187340 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251d751-7c15-4803-8d79-640bcc426bfd service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:04.198437 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1195/2385] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:43:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:43:09.513682 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7937e-d9cd-49c6-9b40-5ed6537af761 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:09.524363 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7937e-d9cd-49c6-9b40-5ed6537af761 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:09.527884 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47acac-0d23-4a9a-b83b-f93795ea60c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:09.528508 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47acac-0d23-4a9a-b83b-f93795ea60c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:09.528788 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47acac-0d23-4a9a-b83b-f93795ea60c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:09.529061 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47acac-0d23-4a9a-b83b-f93795ea60c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:09.531628 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-6f47acac-0d23-4a9a-b83b-f93795ea60c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:09.544677 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47acac-0d23-4a9a-b83b-f93795ea60c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:09.546329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47acac-0d23-4a9a-b83b-f93795ea60c6 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:09.557198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1191/2386] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:43:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:43:14.193530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=6 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:43:17.824850 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251d751-7c15-4803-8d79-640bcc426bfd service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:17.833446 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251d751-7c15-4803-8d79-640bcc426bfd service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:17.837048 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d1b67-7002-4b2d-a08e-7c7f74d58d59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:17.837680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d1b67-7002-4b2d-a08e-7c7f74d58d59 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:17.837963 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d1b67-7002-4b2d-a08e-7c7f74d58d59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:17.838244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d1b67-7002-4b2d-a08e-7c7f74d58d59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:17.840033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4f7d1b67-7002-4b2d-a08e-7c7f74d58d59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:17.902917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d1b67-7002-4b2d-a08e-7c7f74d58d59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:17.904394 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d1b67-7002-4b2d-a08e-7c7f74d58d59 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:17.914712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1196/2387] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:43:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:43:19.552321 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=6 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:43:21.578094 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47acac-0d23-4a9a-b83b-f93795ea60c6 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:21.583135 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-6f47acac-0d23-4a9a-b83b-f93795ea60c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:21.598217 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47acac-0d23-4a9a-b83b-f93795ea60c6 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:21.603501 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56d733-f915-4df9-a2a7-4ebb8e168847 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361be69f1c0a41c18afd206bccf56447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:21.604181 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56d733-f915-4df9-a2a7-4ebb8e168847 service placement] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:21.604469 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56d733-f915-4df9-a2a7-4ebb8e168847 service placement] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:21.604751 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56d733-f915-4df9-a2a7-4ebb8e168847 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:21.606540 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-7a56d733-f915-4df9-a2a7-4ebb8e168847 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:21.619100 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a56d733-f915-4df9-a2a7-4ebb8e168847 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:21.620718 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a56d733-f915-4df9-a2a7-4ebb8e168847 service placement] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:21.640068 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1192/2388] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:43:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:43:21.653502 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d1b67-7002-4b2d-a08e-7c7f74d58d59 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:21.655561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4f7d1b67-7002-4b2d-a08e-7c7f74d58d59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:21.669654 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d1b67-7002-4b2d-a08e-7c7f74d58d59 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:21.675709 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d971da-71ab-4e23-9986-d17ba8ed9f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:21.676629 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d971da-71ab-4e23-9986-d17ba8ed9f25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:21.676910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d971da-71ab-4e23-9986-d17ba8ed9f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:21.677185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d971da-71ab-4e23-9986-d17ba8ed9f25 admin admin] PATH_INFO: `/v3/users/f269e146c844480c822347c3a38d103b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:21.678371 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d971da-71ab-4e23-9986-d17ba8ed9f25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f269e146c844480c822347c3a38d103b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:21.680348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d971da-71ab-4e23-9986-d17ba8ed9f25 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:21.726841 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-24d971da-71ab-4e23-9986-d17ba8ed9f25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f269e146c844480c822347c3a38d103b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:21.734704 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-24d971da-71ab-4e23-9986-d17ba8ed9f25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f269e146c844480c822347c3a38d103b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:21.735375 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-24d971da-71ab-4e23-9986-d17ba8ed9f25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f269e146c844480c822347c3a38d103b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:21.736017 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-24d971da-71ab-4e23-9986-d17ba8ed9f25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f269e146c844480c822347c3a38d103b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:21.747811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-24d971da-71ab-4e23-9986-d17ba8ed9f25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f269e146c844480c822347c3a38d103b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:21.761030 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1197/2389] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:43:21 2022] DELETE /identity/v3/users/f269e146c844480c822347c3a38d103b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:43:21.828074 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56d733-f915-4df9-a2a7-4ebb8e168847 service placement] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:21.837341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56d733-f915-4df9-a2a7-4ebb8e168847 service placement] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:21.888049 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91e996-448c-4c3f-a7a3-19ac7b9cdbc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:21.889225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91e996-448c-4c3f-a7a3-19ac7b9cdbc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:21.889578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91e996-448c-4c3f-a7a3-19ac7b9cdbc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:21.890076 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91e996-448c-4c3f-a7a3-19ac7b9cdbc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:21.892200 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-7b91e996-448c-4c3f-a7a3-19ac7b9cdbc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:21.901759 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91e996-448c-4c3f-a7a3-19ac7b9cdbc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:21.903164 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91e996-448c-4c3f-a7a3-19ac7b9cdbc9 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:21.912406 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1193/2390] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:43:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:43:21.939303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d971da-71ab-4e23-9986-d17ba8ed9f25 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:21.941534 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-24d971da-71ab-4e23-9986-d17ba8ed9f25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:22.014106 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d971da-71ab-4e23-9986-d17ba8ed9f25 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:22.019121 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732696c-fb24-44b3-a6fc-4c8bdfbef4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:22.020029 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732696c-fb24-44b3-a6fc-4c8bdfbef4fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:22.020310 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732696c-fb24-44b3-a6fc-4c8bdfbef4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:22.020577 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732696c-fb24-44b3-a6fc-4c8bdfbef4fd admin admin] PATH_INFO: `/v3/projects/e2b17b61b73b4e6d81dd8ffe9cf311ea` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:22.021886 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732696c-fb24-44b3-a6fc-4c8bdfbef4fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2b17b61b73b4e6d81dd8ffe9cf311ea)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:22.023759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732696c-fb24-44b3-a6fc-4c8bdfbef4fd admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:22.031976 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:43:22.073413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d732696c-fb24-44b3-a6fc-4c8bdfbef4fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2b17b61b73b4e6d81dd8ffe9cf311ea', 'request_id': 'req-d732696c-fb24-44b3-a6fc-4c8bdfbef4fd'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:22.081066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d732696c-fb24-44b3-a6fc-4c8bdfbef4fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2b17b61b73b4e6d81dd8ffe9cf311ea', 'request_id': 'req-d732696c-fb24-44b3-a6fc-4c8bdfbef4fd'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:22.086707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d732696c-fb24-44b3-a6fc-4c8bdfbef4fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2b17b61b73b4e6d81dd8ffe9cf311ea', 'request_id': 'req-d732696c-fb24-44b3-a6fc-4c8bdfbef4fd'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:22.099233 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d732696c-fb24-44b3-a6fc-4c8bdfbef4fd admin admin] The token cache is being invalidate because project e2b17b61b73b4e6d81dd8ffe9cf311ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:43:22.099522 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d732696c-fb24-44b3-a6fc-4c8bdfbef4fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:22.100181 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d732696c-fb24-44b3-a6fc-4c8bdfbef4fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:22.101782 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1198/2391] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:43:21 2022] DELETE /identity/v3/projects/e2b17b61b73b4e6d81dd8ffe9cf311ea => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:43:22.106665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd4666-fa64-484e-a1b4-643492dbc127 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:22.106958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd4666-fa64-484e-a1b4-643492dbc127 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:22.107242 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd4666-fa64-484e-a1b4-643492dbc127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:22.125594 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-f2cd4666-fa64-484e-a1b4-643492dbc127 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:43:22.126794 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-f2cd4666-fa64-484e-a1b4-643492dbc127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:22.189821 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1194/2392] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:43:22.197380 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41a340-7a0f-4162-a83b-78f40e469c18 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:22.197666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41a340-7a0f-4162-a83b-78f40e469c18 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:22.197965 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41a340-7a0f-4162-a83b-78f40e469c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:22.217208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-be41a340-7a0f-4162-a83b-78f40e469c18 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:43:22.218420 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-be41a340-7a0f-4162-a83b-78f40e469c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:22.226939 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1199/2393] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:43:22.231054 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd4666-fa64-484e-a1b4-643492dbc127 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:22.240892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd4666-fa64-484e-a1b4-643492dbc127 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:22.245456 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef1cda-e711-4ba3-8433-25cef804e855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:22.246110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef1cda-e711-4ba3-8433-25cef804e855 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:22.246392 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef1cda-e711-4ba3-8433-25cef804e855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:22.246669 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef1cda-e711-4ba3-8433-25cef804e855 admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:22.247394 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef1cda-e711-4ba3-8433-25cef804e855 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:43:22.247703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef1cda-e711-4ba3-8433-25cef804e855 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:22.249836 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef1cda-e711-4ba3-8433-25cef804e855 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:22.265882 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1195/2394] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:43:22.267034 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41a340-7a0f-4162-a83b-78f40e469c18 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:22.276399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41a340-7a0f-4162-a83b-78f40e469c18 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:22.281250 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca10e1d4-191d-4341-8758-1dfad225ed5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:22.281905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10e1d4-191d-4341-8758-1dfad225ed5d admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:22.282188 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10e1d4-191d-4341-8758-1dfad225ed5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:22.282461 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10e1d4-191d-4341-8758-1dfad225ed5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:22.283140 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca10e1d4-191d-4341-8758-1dfad225ed5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:22.285075 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca10e1d4-191d-4341-8758-1dfad225ed5d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:22.313648 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1200/2395] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:43:22 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:43:22.317841 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef1cda-e711-4ba3-8433-25cef804e855 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:22.328440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef1cda-e711-4ba3-8433-25cef804e855 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:22.397516 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88caafe-eee3-40a0-8a1f-12fabeae1f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:22.398193 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88caafe-eee3-40a0-8a1f-12fabeae1f75 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:22.398645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88caafe-eee3-40a0-8a1f-12fabeae1f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:22.398927 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88caafe-eee3-40a0-8a1f-12fabeae1f75 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:22.400237 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88caafe-eee3-40a0-8a1f-12fabeae1f75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:22.402212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88caafe-eee3-40a0-8a1f-12fabeae1f75 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:22.425443 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1196/2396] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:43:22 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:43:22.430091 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca10e1d4-191d-4341-8758-1dfad225ed5d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:22.442100 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca10e1d4-191d-4341-8758-1dfad225ed5d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:22.447992 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da533cc-685b-4a37-855c-32fd7b2e4659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:22.448686 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da533cc-685b-4a37-855c-32fd7b2e4659 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:22.448973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da533cc-685b-4a37-855c-32fd7b2e4659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:22.449250 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da533cc-685b-4a37-855c-32fd7b2e4659 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:22.450016 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da533cc-685b-4a37-855c-32fd7b2e4659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:22.452357 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da533cc-685b-4a37-855c-32fd7b2e4659 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:22.464575 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1201/2397] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:43:22.469327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88caafe-eee3-40a0-8a1f-12fabeae1f75 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:22.480498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88caafe-eee3-40a0-8a1f-12fabeae1f75 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:22.485612 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab8c184-5036-48e4-b0d0-4b4a88e95fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:22.486341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8c184-5036-48e4-b0d0-4b4a88e95fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:22.486632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8c184-5036-48e4-b0d0-4b4a88e95fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:22.486912 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8c184-5036-48e4-b0d0-4b4a88e95fc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:22.487701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab8c184-5036-48e4-b0d0-4b4a88e95fc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:22.489446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab8c184-5036-48e4-b0d0-4b4a88e95fc6 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:22.491045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:43:22.505172 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1197/2398] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:43:22.509513 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da533cc-685b-4a37-855c-32fd7b2e4659 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:22.521251 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da533cc-685b-4a37-855c-32fd7b2e4659 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:22.526308 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bed467-9b0c-4a36-8aa7-6f7904979d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:22.527322 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bed467-9b0c-4a36-8aa7-6f7904979d0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:22.527669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bed467-9b0c-4a36-8aa7-6f7904979d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:22.528004 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bed467-9b0c-4a36-8aa7-6f7904979d0c admin admin] PATH_INFO: `/v3/projects/92244b4ae17d49bebe621e803bd9ae58/users/7ba341ea1ac34874bbfb8b2f713aea1b/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:22.543737 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bed467-9b0c-4a36-8aa7-6f7904979d0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=92244b4ae17d49bebe621e803bd9ae58, user_id=7ba341ea1ac34874bbfb8b2f713aea1b, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:22.547012 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bed467-9b0c-4a36-8aa7-6f7904979d0c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:22.561454 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1202/2399] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:43:22 2022] PUT /identity/v3/projects/92244b4ae17d49bebe621e803bd9ae58/users/7ba341ea1ac34874bbfb8b2f713aea1b/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:43:22.566827 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98a5ff-feca-4c34-80c6-4ae4e0cb514d None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:22.567116 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98a5ff-feca-4c34-80c6-4ae4e0cb514d None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:22.567467 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98a5ff-feca-4c34-80c6-4ae4e0cb514d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:22.586337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-2c98a5ff-feca-4c34-80c6-4ae4e0cb514d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:43:22.588297 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-2c98a5ff-feca-4c34-80c6-4ae4e0cb514d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:22.654071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1198/2400] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:43:22.663992 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bed467-9b0c-4a36-8aa7-6f7904979d0c admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:22.666058 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-54bed467-9b0c-4a36-8aa7-6f7904979d0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:22.725267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bed467-9b0c-4a36-8aa7-6f7904979d0c admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:22.728993 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475d1dc-a05d-4375-a739-8bee1bea9e4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:22.729621 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475d1dc-a05d-4375-a739-8bee1bea9e4a service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:22.729901 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475d1dc-a05d-4375-a739-8bee1bea9e4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:22.730175 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475d1dc-a05d-4375-a739-8bee1bea9e4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:22.740001 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475d1dc-a05d-4375-a739-8bee1bea9e4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:22.741411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475d1dc-a05d-4375-a739-8bee1bea9e4a service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:22.751700 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1203/2401] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:43:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:43:25.620408 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98a5ff-feca-4c34-80c6-4ae4e0cb514d None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:25.625391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-2c98a5ff-feca-4c34-80c6-4ae4e0cb514d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:25.790127 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98a5ff-feca-4c34-80c6-4ae4e0cb514d None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:25.812505 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4755099-7ea2-4774-819e-6788652d6685 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:25.813371 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4755099-7ea2-4774-819e-6788652d6685 service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:25.813654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4755099-7ea2-4774-819e-6788652d6685 service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:25.813927 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4755099-7ea2-4774-819e-6788652d6685 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:25.817702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-d4755099-7ea2-4774-819e-6788652d6685 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:25.854077 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4755099-7ea2-4774-819e-6788652d6685 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:25.855695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4755099-7ea2-4774-819e-6788652d6685 service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:25.887607 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1199/2402] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:43:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:43:26.401084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475d1dc-a05d-4375-a739-8bee1bea9e4a service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:26.403509 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1475d1dc-a05d-4375-a739-8bee1bea9e4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:26.457055 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:43:26.549857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475d1dc-a05d-4375-a739-8bee1bea9e4a service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:26.563819 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd7fb4-9f2f-4ff1-a6e5-f49cb0b35dbf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361be69f1c0a41c18afd206bccf56447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:26.564663 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd7fb4-9f2f-4ff1-a6e5-f49cb0b35dbf service placement] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:26.565046 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd7fb4-9f2f-4ff1-a6e5-f49cb0b35dbf service placement] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:26.565430 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd7fb4-9f2f-4ff1-a6e5-f49cb0b35dbf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:26.570656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a9dd7fb4-9f2f-4ff1-a6e5-f49cb0b35dbf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:26.590800 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd7fb4-9f2f-4ff1-a6e5-f49cb0b35dbf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:26.594551 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd7fb4-9f2f-4ff1-a6e5-f49cb0b35dbf service placement] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:26.606999 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1204/2403] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:43:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:43:30.038758 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585ed3b-63a6-4b72-9816-257e22f4f10b None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:30.039636 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585ed3b-63a6-4b72-9816-257e22f4f10b None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:30.040092 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585ed3b-63a6-4b72-9816-257e22f4f10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:30.072855 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-b585ed3b-63a6-4b72-9816-257e22f4f10b None None] MFA Rules not processed for user `7ba341ea1ac34874bbfb8b2f713aea1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:43:30.074133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-b585ed3b-63a6-4b72-9816-257e22f4f10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:30.159973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1200/2404] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:43:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:43:30.174717 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd7fb4-9f2f-4ff1-a6e5-f49cb0b35dbf service placement] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:30.176289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a9dd7fb4-9f2f-4ff1-a6e5-f49cb0b35dbf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:30.192653 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd7fb4-9f2f-4ff1-a6e5-f49cb0b35dbf service placement] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:30.198290 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482d8d3-331f-4675-85bb-843b47bb69a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:30.198937 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482d8d3-331f-4675-85bb-843b47bb69a7 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:30.199222 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482d8d3-331f-4675-85bb-843b47bb69a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:30.199528 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482d8d3-331f-4675-85bb-843b47bb69a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:30.208264 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482d8d3-331f-4675-85bb-843b47bb69a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:30.209852 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482d8d3-331f-4675-85bb-843b47bb69a7 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:30.219368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1205/2405] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:43:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:43:31.868529 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc2a7c-b181-43d5-b642-67a4c96ea4c7 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:31.869090 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc2a7c-b181-43d5-b642-67a4c96ea4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:31.869090 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc2a7c-b181-43d5-b642-67a4c96ea4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:31.889808 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-a5dc2a7c-b181-43d5-b642-67a4c96ea4c7 None None] MFA Rules not processed for user `7ba341ea1ac34874bbfb8b2f713aea1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:43:31.894023 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a5dc2a7c-b181-43d5-b642-67a4c96ea4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:31.908850 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1201/2406] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:43:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:43:32.264381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482d8d3-331f-4675-85bb-843b47bb69a7 service nova] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:32.266073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2482d8d3-331f-4675-85bb-843b47bb69a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:32.278301 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482d8d3-331f-4675-85bb-843b47bb69a7 service nova] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:32.282827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00651f34-28bf-4c77-be3c-d7b86a207889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:32.283484 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00651f34-28bf-4c77-be3c-d7b86a207889 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:32.283769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00651f34-28bf-4c77-be3c-d7b86a207889 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:32.284081 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00651f34-28bf-4c77-be3c-d7b86a207889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:32.286032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-00651f34-28bf-4c77-be3c-d7b86a207889 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:32.296577 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00651f34-28bf-4c77-be3c-d7b86a207889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:32.298070 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00651f34-28bf-4c77-be3c-d7b86a207889 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:32.307755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1206/2407] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:43:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:43:33.472161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc2a7c-b181-43d5-b642-67a4c96ea4c7 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:33.480434 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:43:33.486550 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc2a7c-b181-43d5-b642-67a4c96ea4c7 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:33.492062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173b38a-d2f1-4f4c-8af9-76defbd80e46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:33.492698 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173b38a-d2f1-4f4c-8af9-76defbd80e46 service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:33.493056 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173b38a-d2f1-4f4c-8af9-76defbd80e46 service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:33.493339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173b38a-d2f1-4f4c-8af9-76defbd80e46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:33.495080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-c173b38a-d2f1-4f4c-8af9-76defbd80e46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:33.568105 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c173b38a-d2f1-4f4c-8af9-76defbd80e46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:33.569595 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c173b38a-d2f1-4f4c-8af9-76defbd80e46 service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:33.579617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1202/2408] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:43:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:43:35.045365 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354bf24-3b4a-4365-9383-b258fdb373be None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:35.045853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354bf24-3b4a-4365-9383-b258fdb373be None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:35.046044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354bf24-3b4a-4365-9383-b258fdb373be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:35.070784 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9354bf24-3b4a-4365-9383-b258fdb373be None None] MFA Rules not processed for user `e9e358afe0974c0c8bd53104ebbff445`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:43:35.072189 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9354bf24-3b4a-4365-9383-b258fdb373be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:35.133272 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1207/2409] 10.222.0.42 () {58 vars in 967 bytes} [Mon Jun 6 10:43:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:43:35.184107 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173b38a-d2f1-4f4c-8af9-76defbd80e46 service nova] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:35.186108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-c173b38a-d2f1-4f4c-8af9-76defbd80e46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:35.267024 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173b38a-d2f1-4f4c-8af9-76defbd80e46 service nova] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:35.272206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b07a4-4a52-4f9e-bcf2-90738af00f99 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fede9e52ad274285b09d748a684ede38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:35.272857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b07a4-4a52-4f9e-bcf2-90738af00f99 service swift] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:35.273181 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b07a4-4a52-4f9e-bcf2-90738af00f99 service swift] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:35.273485 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b07a4-4a52-4f9e-bcf2-90738af00f99 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:35.283712 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b07a4-4a52-4f9e-bcf2-90738af00f99 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:35.285338 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b07a4-4a52-4f9e-bcf2-90738af00f99 service swift] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:35.300133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1203/2410] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jun 6 10:43:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 10:43:36.395891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354bf24-3b4a-4365-9383-b258fdb373be None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:36.409247 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354bf24-3b4a-4365-9383-b258fdb373be None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:36.425083 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fa483-5c3f-49c1-acfb-a63a0d252d6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:36.425833 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fa483-5c3f-49c1-acfb-a63a0d252d6d service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:36.426317 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fa483-5c3f-49c1-acfb-a63a0d252d6d service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:36.426792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fa483-5c3f-49c1-acfb-a63a0d252d6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:36.428827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c52fa483-5c3f-49c1-acfb-a63a0d252d6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:36.441746 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52fa483-5c3f-49c1-acfb-a63a0d252d6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:36.443409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52fa483-5c3f-49c1-acfb-a63a0d252d6d service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:36.454644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1208/2411] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:43:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:43:41.451236 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b07a4-4a52-4f9e-bcf2-90738af00f99 service swift] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:41.464372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b07a4-4a52-4f9e-bcf2-90738af00f99 service swift] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:41.470935 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f23c7-b52a-426c-b787-0464d0661341 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:41.471686 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f23c7-b52a-426c-b787-0464d0661341 service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:41.472068 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f23c7-b52a-426c-b787-0464d0661341 service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:41.472411 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f23c7-b52a-426c-b787-0464d0661341 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:41.474434 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-7a6f23c7-b52a-426c-b787-0464d0661341 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:41.559514 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f23c7-b52a-426c-b787-0464d0661341 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:41.561942 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f23c7-b52a-426c-b787-0464d0661341 service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:41.572884 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1204/2412] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:43:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:43:42.751824 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fa483-5c3f-49c1-acfb-a63a0d252d6d service nova] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:42.753368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c52fa483-5c3f-49c1-acfb-a63a0d252d6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:42.770446 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fa483-5c3f-49c1-acfb-a63a0d252d6d service nova] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:42.775043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e80c47-b35f-4ce9-8b37-a2fa8e488a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:42.775932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e80c47-b35f-4ce9-8b37-a2fa8e488a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:42.776212 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e80c47-b35f-4ce9-8b37-a2fa8e488a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:42.776484 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e80c47-b35f-4ce9-8b37-a2fa8e488a3f admin admin] PATH_INFO: `/v3/users/b7b11ed004fc4ef3a1f1c48adf31fe8e` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:42.777645 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e80c47-b35f-4ce9-8b37-a2fa8e488a3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7b11ed004fc4ef3a1f1c48adf31fe8e)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:42.779587 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e80c47-b35f-4ce9-8b37-a2fa8e488a3f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:42.826045 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f5e80c47-b35f-4ce9-8b37-a2fa8e488a3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7b11ed004fc4ef3a1f1c48adf31fe8e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:42.832793 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f5e80c47-b35f-4ce9-8b37-a2fa8e488a3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7b11ed004fc4ef3a1f1c48adf31fe8e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:42.833452 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f5e80c47-b35f-4ce9-8b37-a2fa8e488a3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7b11ed004fc4ef3a1f1c48adf31fe8e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:42.834063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f5e80c47-b35f-4ce9-8b37-a2fa8e488a3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7b11ed004fc4ef3a1f1c48adf31fe8e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:42.845788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f5e80c47-b35f-4ce9-8b37-a2fa8e488a3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7b11ed004fc4ef3a1f1c48adf31fe8e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:42.860887 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1209/2413] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:43:42 2022] DELETE /identity/v3/users/b7b11ed004fc4ef3a1f1c48adf31fe8e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:43:43.047863 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f23c7-b52a-426c-b787-0464d0661341 service nova] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:43.064493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f23c7-b52a-426c-b787-0464d0661341 service nova] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:43.097603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:43:43.140783 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a32a0-72cf-457c-8198-5477e838aacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:43.141648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a32a0-72cf-457c-8198-5477e838aacb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:43.141928 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a32a0-72cf-457c-8198-5477e838aacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:43.142207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a32a0-72cf-457c-8198-5477e838aacb admin admin] PATH_INFO: `/v3/projects/645eadf589944a24bf6e27326a61952c` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:43.143388 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a32a0-72cf-457c-8198-5477e838aacb admin admin] RBAC: Authorizing `identity:delete_project(project_id=645eadf589944a24bf6e27326a61952c)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:43.145313 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a32a0-72cf-457c-8198-5477e838aacb admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:43.202510 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-841a32a0-72cf-457c-8198-5477e838aacb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '645eadf589944a24bf6e27326a61952c', 'request_id': 'req-841a32a0-72cf-457c-8198-5477e838aacb'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:43.208873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-841a32a0-72cf-457c-8198-5477e838aacb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '645eadf589944a24bf6e27326a61952c', 'request_id': 'req-841a32a0-72cf-457c-8198-5477e838aacb'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:43.221311 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-841a32a0-72cf-457c-8198-5477e838aacb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '645eadf589944a24bf6e27326a61952c', 'request_id': 'req-841a32a0-72cf-457c-8198-5477e838aacb'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:43.235689 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-841a32a0-72cf-457c-8198-5477e838aacb admin admin] The token cache is being invalidate because project 645eadf589944a24bf6e27326a61952c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:43:43.236012 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-841a32a0-72cf-457c-8198-5477e838aacb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:43.236669 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-841a32a0-72cf-457c-8198-5477e838aacb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:43.242396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1205/2414] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:43:43 2022] DELETE /identity/v3/projects/645eadf589944a24bf6e27326a61952c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:43:43.243437 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e80c47-b35f-4ce9-8b37-a2fa8e488a3f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:43.245019 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f5e80c47-b35f-4ce9-8b37-a2fa8e488a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:43.254506 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:43:43.260279 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e80c47-b35f-4ce9-8b37-a2fa8e488a3f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:43.336640 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c1481-b601-4b15-acd5-c9c12f782c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:43.337621 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c1481-b601-4b15-acd5-c9c12f782c7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:43.338010 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c1481-b601-4b15-acd5-c9c12f782c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:43.338173 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c1481-b601-4b15-acd5-c9c12f782c7b admin admin] PATH_INFO: `/v3/users/f791934803044943b030f1f1677e2361` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:43.339952 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c1481-b601-4b15-acd5-c9c12f782c7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f791934803044943b030f1f1677e2361)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:43.344226 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c1481-b601-4b15-acd5-c9c12f782c7b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:43.402666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-649c1481-b601-4b15-acd5-c9c12f782c7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f791934803044943b030f1f1677e2361'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:43.413678 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-649c1481-b601-4b15-acd5-c9c12f782c7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f791934803044943b030f1f1677e2361'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:43.414442 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-649c1481-b601-4b15-acd5-c9c12f782c7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f791934803044943b030f1f1677e2361'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:43.415767 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-649c1481-b601-4b15-acd5-c9c12f782c7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f791934803044943b030f1f1677e2361'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:43.430482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-649c1481-b601-4b15-acd5-c9c12f782c7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f791934803044943b030f1f1677e2361'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:43.446066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1210/2415] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:43:43 2022] DELETE /identity/v3/users/f791934803044943b030f1f1677e2361 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:43:43.726823 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a32a0-72cf-457c-8198-5477e838aacb admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:43.728286 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-841a32a0-72cf-457c-8198-5477e838aacb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:43.741316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a32a0-72cf-457c-8198-5477e838aacb admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:43.801773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2f29d-78ee-4f1a-bd02-8aafcd028e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:43.802601 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2f29d-78ee-4f1a-bd02-8aafcd028e9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:43.802885 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2f29d-78ee-4f1a-bd02-8aafcd028e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:43.803165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2f29d-78ee-4f1a-bd02-8aafcd028e9e admin admin] PATH_INFO: `/v3/projects/2dbd010f8c024d7dac26979caa814c4e` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:43.804499 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e2f29d-78ee-4f1a-bd02-8aafcd028e9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dbd010f8c024d7dac26979caa814c4e)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:43.806573 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e2f29d-78ee-4f1a-bd02-8aafcd028e9e admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:43.855692 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-36e2f29d-78ee-4f1a-bd02-8aafcd028e9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dbd010f8c024d7dac26979caa814c4e', 'request_id': 'req-36e2f29d-78ee-4f1a-bd02-8aafcd028e9e'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:43.861491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-36e2f29d-78ee-4f1a-bd02-8aafcd028e9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dbd010f8c024d7dac26979caa814c4e', 'request_id': 'req-36e2f29d-78ee-4f1a-bd02-8aafcd028e9e'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:43.865549 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:43:43.870239 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-36e2f29d-78ee-4f1a-bd02-8aafcd028e9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dbd010f8c024d7dac26979caa814c4e', 'request_id': 'req-36e2f29d-78ee-4f1a-bd02-8aafcd028e9e'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:43.884567 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-36e2f29d-78ee-4f1a-bd02-8aafcd028e9e admin admin] The token cache is being invalidate because project 2dbd010f8c024d7dac26979caa814c4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:43:43.884859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-36e2f29d-78ee-4f1a-bd02-8aafcd028e9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:43.885515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-36e2f29d-78ee-4f1a-bd02-8aafcd028e9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:43:43.887163 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1206/2416] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:43:43 2022] DELETE /identity/v3/projects/2dbd010f8c024d7dac26979caa814c4e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:43:44.788592 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c1481-b601-4b15-acd5-c9c12f782c7b admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:43:44.790260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-649c1481-b601-4b15-acd5-c9c12f782c7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:44.867462 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c1481-b601-4b15-acd5-c9c12f782c7b admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:43:44.884914 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:43:44.937284 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d036e-43ea-4d3e-97da-3a54af369111 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:43:44.937941 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d036e-43ea-4d3e-97da-3a54af369111 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:43:44.938219 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d036e-43ea-4d3e-97da-3a54af369111 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:43:44.938491 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d036e-43ea-4d3e-97da-3a54af369111 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:43:44.940293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4f1d036e-43ea-4d3e-97da-3a54af369111 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:43:44.952419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d036e-43ea-4d3e-97da-3a54af369111 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:43:44.954119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d036e-43ea-4d3e-97da-3a54af369111 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:43:44.964786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1211/2417] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:43:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:43:53.880916 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=4 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:43:54.960364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=25 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:00.581836 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e79312-49b4-40a7-bb5c-931ee335111e None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:00.582302 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e79312-49b4-40a7-bb5c-931ee335111e None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:00.582407 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e79312-49b4-40a7-bb5c-931ee335111e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:00.600709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-23e79312-49b4-40a7-bb5c-931ee335111e None None] MFA Rules not processed for user `e9e358afe0974c0c8bd53104ebbff445`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:00.602160 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-23e79312-49b4-40a7-bb5c-931ee335111e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:00.661713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1207/2418] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jun 6 10:44:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:00.674829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d036e-43ea-4d3e-97da-3a54af369111 service nova] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:00.676251 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4f1d036e-43ea-4d3e-97da-3a54af369111 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:00.744590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d036e-43ea-4d3e-97da-3a54af369111 service nova] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:00.750474 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f516d-76e3-4e90-a9c8-f690ed2cf7e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fede9e52ad274285b09d748a684ede38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:00.751131 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f516d-76e3-4e90-a9c8-f690ed2cf7e9 service swift] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:00.751442 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f516d-76e3-4e90-a9c8-f690ed2cf7e9 service swift] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:00.751866 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f516d-76e3-4e90-a9c8-f690ed2cf7e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:00.760421 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f516d-76e3-4e90-a9c8-f690ed2cf7e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:00.762257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f516d-76e3-4e90-a9c8-f690ed2cf7e9 service swift] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:00.770808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1212/2419] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jun 6 10:44:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 10:44:10.654980 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=25 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:10.767465 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=32 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:13.376303 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e79312-49b4-40a7-bb5c-931ee335111e None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:13.377811 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-23e79312-49b4-40a7-bb5c-931ee335111e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:13.433287 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e79312-49b4-40a7-bb5c-931ee335111e None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:13.437707 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bac38-95ae-49a9-9405-f6f156c58c8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:13.438345 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bac38-95ae-49a9-9405-f6f156c58c8a service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:13.438648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bac38-95ae-49a9-9405-f6f156c58c8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:13.438908 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bac38-95ae-49a9-9405-f6f156c58c8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:13.440697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ef2bac38-95ae-49a9-9405-f6f156c58c8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:13.502201 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2bac38-95ae-49a9-9405-f6f156c58c8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:13.503544 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2bac38-95ae-49a9-9405-f6f156c58c8a service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:13.513280 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1208/2420] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:44:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:44:16.998215 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098db0e-88d0-41be-87ff-43ea215da809 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:16.998758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098db0e-88d0-41be-87ff-43ea215da809 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:16.998758 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098db0e-88d0-41be-87ff-43ea215da809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:17.029979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-d098db0e-88d0-41be-87ff-43ea215da809 None None] MFA Rules not processed for user `e9e358afe0974c0c8bd53104ebbff445`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:17.031256 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d098db0e-88d0-41be-87ff-43ea215da809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:17.040147 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1213/2421] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jun 6 10:44:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:17.063739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bac38-95ae-49a9-9405-f6f156c58c8a service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:17.085058 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bac38-95ae-49a9-9405-f6f156c58c8a service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:17.096225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aadbff-7539-47ea-b386-41fea703c664 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fede9e52ad274285b09d748a684ede38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:17.097026 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aadbff-7539-47ea-b386-41fea703c664 service swift] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:17.097740 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aadbff-7539-47ea-b386-41fea703c664 service swift] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:17.098137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aadbff-7539-47ea-b386-41fea703c664 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:17.108469 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aadbff-7539-47ea-b386-41fea703c664 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:17.110118 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aadbff-7539-47ea-b386-41fea703c664 service swift] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:17.120240 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1209/2422] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jun 6 10:44:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 10:44:17.264113 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098db0e-88d0-41be-87ff-43ea215da809 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:17.265878 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d098db0e-88d0-41be-87ff-43ea215da809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:17.311611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098db0e-88d0-41be-87ff-43ea215da809 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:17.325042 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695205cb-2138-4341-9cc0-05776a3235ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:17.326106 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695205cb-2138-4341-9cc0-05776a3235ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:17.326386 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695205cb-2138-4341-9cc0-05776a3235ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:17.326658 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695205cb-2138-4341-9cc0-05776a3235ea admin admin] PATH_INFO: `/v3/users/f6b7d82c999646b6b6b96d565f0e3f9a` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:17.328580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695205cb-2138-4341-9cc0-05776a3235ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6b7d82c999646b6b6b96d565f0e3f9a)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:17.332870 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695205cb-2138-4341-9cc0-05776a3235ea admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:17.436574 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-695205cb-2138-4341-9cc0-05776a3235ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6b7d82c999646b6b6b96d565f0e3f9a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:17.448251 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-695205cb-2138-4341-9cc0-05776a3235ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6b7d82c999646b6b6b96d565f0e3f9a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:17.450293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-695205cb-2138-4341-9cc0-05776a3235ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6b7d82c999646b6b6b96d565f0e3f9a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:17.451230 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-695205cb-2138-4341-9cc0-05776a3235ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6b7d82c999646b6b6b96d565f0e3f9a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:17.465823 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-695205cb-2138-4341-9cc0-05776a3235ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6b7d82c999646b6b6b96d565f0e3f9a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:17.492052 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1214/2423] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:44:17 2022] DELETE /identity/v3/users/f6b7d82c999646b6b6b96d565f0e3f9a => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:17.870520 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aadbff-7539-47ea-b386-41fea703c664 service swift] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:17.885138 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aadbff-7539-47ea-b386-41fea703c664 service swift] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:17.977480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfee71-2f2d-4da4-934e-86f7c498e0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:17.978512 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfee71-2f2d-4da4-934e-86f7c498e0e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:17.978799 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfee71-2f2d-4da4-934e-86f7c498e0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:17.979233 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfee71-2f2d-4da4-934e-86f7c498e0e9 admin admin] PATH_INFO: `/v3/projects/fe30c756a2f84166ad23af1edf83157e` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:17.980960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cfee71-2f2d-4da4-934e-86f7c498e0e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe30c756a2f84166ad23af1edf83157e)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:17.982896 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cfee71-2f2d-4da4-934e-86f7c498e0e9 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:18.034996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:18.049716 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-30cfee71-2f2d-4da4-934e-86f7c498e0e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe30c756a2f84166ad23af1edf83157e', 'request_id': 'req-30cfee71-2f2d-4da4-934e-86f7c498e0e9'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:18.056936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-30cfee71-2f2d-4da4-934e-86f7c498e0e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe30c756a2f84166ad23af1edf83157e', 'request_id': 'req-30cfee71-2f2d-4da4-934e-86f7c498e0e9'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:18.068765 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-30cfee71-2f2d-4da4-934e-86f7c498e0e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe30c756a2f84166ad23af1edf83157e', 'request_id': 'req-30cfee71-2f2d-4da4-934e-86f7c498e0e9'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:18.082949 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-30cfee71-2f2d-4da4-934e-86f7c498e0e9 admin admin] The token cache is being invalidate because project fe30c756a2f84166ad23af1edf83157e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:44:18.083263 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-30cfee71-2f2d-4da4-934e-86f7c498e0e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:18.083955 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-30cfee71-2f2d-4da4-934e-86f7c498e0e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:18.085815 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1210/2424] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:44:17 2022] DELETE /identity/v3/projects/fe30c756a2f84166ad23af1edf83157e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:18.091208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531952c5-3644-4774-a60e-1386c1d8f200 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:18.091707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531952c5-3644-4774-a60e-1386c1d8f200 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:18.091838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531952c5-3644-4774-a60e-1386c1d8f200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:18.112902 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-531952c5-3644-4774-a60e-1386c1d8f200 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:18.114116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-531952c5-3644-4774-a60e-1386c1d8f200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:18.179241 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1215/2425] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:18.188273 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d250fa3-7016-4d1c-94ca-ff7b34f9a1a3 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:18.188799 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d250fa3-7016-4d1c-94ca-ff7b34f9a1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:18.189079 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d250fa3-7016-4d1c-94ca-ff7b34f9a1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:18.211094 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-1d250fa3-7016-4d1c-94ca-ff7b34f9a1a3 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:18.212339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-1d250fa3-7016-4d1c-94ca-ff7b34f9a1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:18.220897 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1211/2426] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:18.225112 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531952c5-3644-4774-a60e-1386c1d8f200 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:18.235022 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531952c5-3644-4774-a60e-1386c1d8f200 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:18.239558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cc3bf-bafa-41f3-850c-64772ec8713c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:18.240330 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cc3bf-bafa-41f3-850c-64772ec8713c admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:18.240611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cc3bf-bafa-41f3-850c-64772ec8713c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:18.240881 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cc3bf-bafa-41f3-850c-64772ec8713c admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:18.241596 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49cc3bf-bafa-41f3-850c-64772ec8713c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:44:18.241897 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49cc3bf-bafa-41f3-850c-64772ec8713c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:18.243481 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49cc3bf-bafa-41f3-850c-64772ec8713c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:18.255376 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1216/2427] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:44:18.259635 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d250fa3-7016-4d1c-94ca-ff7b34f9a1a3 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:18.273551 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d250fa3-7016-4d1c-94ca-ff7b34f9a1a3 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:18.278875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c84cd-35e4-4fcf-ae78-3b8076d11c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:18.279615 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c84cd-35e4-4fcf-ae78-3b8076d11c6a admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:18.279929 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c84cd-35e4-4fcf-ae78-3b8076d11c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:18.280208 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c84cd-35e4-4fcf-ae78-3b8076d11c6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:18.280914 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c84cd-35e4-4fcf-ae78-3b8076d11c6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:18.282821 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c84cd-35e4-4fcf-ae78-3b8076d11c6a admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:18.313866 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1212/2428] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:44:18 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:44:18.318020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cc3bf-bafa-41f3-850c-64772ec8713c admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:18.328748 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cc3bf-bafa-41f3-850c-64772ec8713c admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:18.395571 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:18.406829 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe6dc5-ed1e-46be-a4d2-b2ff54f1d343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:18.407555 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe6dc5-ed1e-46be-a4d2-b2ff54f1d343 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:18.407868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe6dc5-ed1e-46be-a4d2-b2ff54f1d343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:18.408144 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe6dc5-ed1e-46be-a4d2-b2ff54f1d343 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:18.408833 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe6dc5-ed1e-46be-a4d2-b2ff54f1d343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:18.410776 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe6dc5-ed1e-46be-a4d2-b2ff54f1d343 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:18.431256 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1217/2429] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:44:18 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:44:18.436076 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c84cd-35e4-4fcf-ae78-3b8076d11c6a admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:18.447933 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c84cd-35e4-4fcf-ae78-3b8076d11c6a admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:18.452941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bd150-c791-46f8-896f-df7e07ad8192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:18.453620 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bd150-c791-46f8-896f-df7e07ad8192 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:18.453908 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bd150-c791-46f8-896f-df7e07ad8192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:18.454188 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bd150-c791-46f8-896f-df7e07ad8192 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:18.454920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5bd150-c791-46f8-896f-df7e07ad8192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:18.456587 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5bd150-c791-46f8-896f-df7e07ad8192 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:18.469509 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1213/2430] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:18.472774 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe6dc5-ed1e-46be-a4d2-b2ff54f1d343 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:18.485241 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe6dc5-ed1e-46be-a4d2-b2ff54f1d343 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:18.490224 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aba400-c787-4716-9c73-a26beac96c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:18.490890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aba400-c787-4716-9c73-a26beac96c62 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:18.491205 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aba400-c787-4716-9c73-a26beac96c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:18.491483 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aba400-c787-4716-9c73-a26beac96c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:18.492283 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aba400-c787-4716-9c73-a26beac96c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:18.493894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aba400-c787-4716-9c73-a26beac96c62 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:18.506440 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1218/2431] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:18.510769 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bd150-c791-46f8-896f-df7e07ad8192 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:18.521160 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bd150-c791-46f8-896f-df7e07ad8192 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:18.526005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e109ed-b4ef-44f9-9e41-e06069449ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:18.526979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e109ed-b4ef-44f9-9e41-e06069449ee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:18.527327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e109ed-b4ef-44f9-9e41-e06069449ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:18.527604 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e109ed-b4ef-44f9-9e41-e06069449ee9 admin admin] PATH_INFO: `/v3/projects/67054e1e988a4691a3c8a8816c21de82/users/8972281e0c934d398104d8c78ebfdf71/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:18.543044 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e109ed-b4ef-44f9-9e41-e06069449ee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67054e1e988a4691a3c8a8816c21de82, user_id=8972281e0c934d398104d8c78ebfdf71, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:18.545740 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e109ed-b4ef-44f9-9e41-e06069449ee9 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:18.564356 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1214/2432] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:44:18 2022] PUT /identity/v3/projects/67054e1e988a4691a3c8a8816c21de82/users/8972281e0c934d398104d8c78ebfdf71/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:18.578138 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581e314-6958-413d-867c-fcbe40f67204 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:18.578138 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581e314-6958-413d-867c-fcbe40f67204 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:18.578138 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581e314-6958-413d-867c-fcbe40f67204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:18.610067 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-e581e314-6958-413d-867c-fcbe40f67204 None None] MFA Rules not processed for user `8972281e0c934d398104d8c78ebfdf71`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:18.611328 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e581e314-6958-413d-867c-fcbe40f67204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:18.686349 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1219/2433] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:18.697410 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e109ed-b4ef-44f9-9e41-e06069449ee9 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:18.699419 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-54e109ed-b4ef-44f9-9e41-e06069449ee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:18.798546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e109ed-b4ef-44f9-9e41-e06069449ee9 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:18.806261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc7686-e1bb-4672-b3c4-d48c7438fc13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:18.806923 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc7686-e1bb-4672-b3c4-d48c7438fc13 service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:18.807207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc7686-e1bb-4672-b3c4-d48c7438fc13 service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:18.807482 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc7686-e1bb-4672-b3c4-d48c7438fc13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:18.816409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc7686-e1bb-4672-b3c4-d48c7438fc13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:18.817897 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc7686-e1bb-4672-b3c4-d48c7438fc13 service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:18.835005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1215/2434] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:44:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:44:21.702524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581e314-6958-413d-867c-fcbe40f67204 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:21.772446 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581e314-6958-413d-867c-fcbe40f67204 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:21.778243 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290592de-cee3-44d1-af3d-033e141edffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:21.779125 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290592de-cee3-44d1-af3d-033e141edffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:21.779403 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290592de-cee3-44d1-af3d-033e141edffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:21.779673 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290592de-cee3-44d1-af3d-033e141edffd admin admin] PATH_INFO: `/v3/users/8972281e0c934d398104d8c78ebfdf71` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:21.781330 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290592de-cee3-44d1-af3d-033e141edffd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8972281e0c934d398104d8c78ebfdf71)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:21.783257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290592de-cee3-44d1-af3d-033e141edffd admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:21.834225 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-290592de-cee3-44d1-af3d-033e141edffd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8972281e0c934d398104d8c78ebfdf71'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:21.841320 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-290592de-cee3-44d1-af3d-033e141edffd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8972281e0c934d398104d8c78ebfdf71'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:21.841981 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-290592de-cee3-44d1-af3d-033e141edffd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8972281e0c934d398104d8c78ebfdf71'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:21.842584 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-290592de-cee3-44d1-af3d-033e141edffd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8972281e0c934d398104d8c78ebfdf71'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:21.849251 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:21.860606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-290592de-cee3-44d1-af3d-033e141edffd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8972281e0c934d398104d8c78ebfdf71'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:21.879041 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1220/2435] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:44:21 2022] DELETE /identity/v3/users/8972281e0c934d398104d8c78ebfdf71 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:21.888429 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99fbe8-4ebb-48d3-815a-1dfd8adf905d None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:21.891301 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99fbe8-4ebb-48d3-815a-1dfd8adf905d None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:21.891301 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99fbe8-4ebb-48d3-815a-1dfd8adf905d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:21.913447 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-6a99fbe8-4ebb-48d3-815a-1dfd8adf905d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:21.914695 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-6a99fbe8-4ebb-48d3-815a-1dfd8adf905d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:21.964996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:21.982100 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1216/2436] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:21.991764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290592de-cee3-44d1-af3d-033e141edffd admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:21.993167 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-290592de-cee3-44d1-af3d-033e141edffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:22.056780 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290592de-cee3-44d1-af3d-033e141edffd admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:22.062947 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe4748-9fec-4357-911b-01b84a3ae3ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:22.063756 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe4748-9fec-4357-911b-01b84a3ae3ac service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:22.064065 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe4748-9fec-4357-911b-01b84a3ae3ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:22.064339 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe4748-9fec-4357-911b-01b84a3ae3ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:22.076110 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe4748-9fec-4357-911b-01b84a3ae3ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:22.077562 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe4748-9fec-4357-911b-01b84a3ae3ac service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:22.088507 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1221/2437] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:44:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:44:22.441825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99fbe8-4ebb-48d3-815a-1dfd8adf905d None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:22.443568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-6a99fbe8-4ebb-48d3-815a-1dfd8adf905d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:22.469017 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99fbe8-4ebb-48d3-815a-1dfd8adf905d None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:22.477910 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d9e91-581d-4bdb-b493-89d846d64ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:22.478795 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d9e91-581d-4bdb-b493-89d846d64ce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:22.479189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d9e91-581d-4bdb-b493-89d846d64ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:22.479482 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d9e91-581d-4bdb-b493-89d846d64ce2 admin admin] PATH_INFO: `/v3/projects/67054e1e988a4691a3c8a8816c21de82` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:22.482284 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d9e91-581d-4bdb-b493-89d846d64ce2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67054e1e988a4691a3c8a8816c21de82)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:22.484715 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d9e91-581d-4bdb-b493-89d846d64ce2 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:22.556002 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1c7d9e91-581d-4bdb-b493-89d846d64ce2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67054e1e988a4691a3c8a8816c21de82', 'request_id': 'req-1c7d9e91-581d-4bdb-b493-89d846d64ce2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:22.562285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1c7d9e91-581d-4bdb-b493-89d846d64ce2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67054e1e988a4691a3c8a8816c21de82', 'request_id': 'req-1c7d9e91-581d-4bdb-b493-89d846d64ce2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:22.570716 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1c7d9e91-581d-4bdb-b493-89d846d64ce2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67054e1e988a4691a3c8a8816c21de82', 'request_id': 'req-1c7d9e91-581d-4bdb-b493-89d846d64ce2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:22.583571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1c7d9e91-581d-4bdb-b493-89d846d64ce2 admin admin] The token cache is being invalidate because project 67054e1e988a4691a3c8a8816c21de82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:44:22.583915 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1c7d9e91-581d-4bdb-b493-89d846d64ce2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:22.584610 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1c7d9e91-581d-4bdb-b493-89d846d64ce2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:22.586792 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1217/2438] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:44:22 2022] DELETE /identity/v3/projects/67054e1e988a4691a3c8a8816c21de82 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:22.592854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3500a0f-67dc-4aab-bc60-c5c52c76e5db None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:22.593282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3500a0f-67dc-4aab-bc60-c5c52c76e5db None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:22.593655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3500a0f-67dc-4aab-bc60-c5c52c76e5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:22.618083 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b3500a0f-67dc-4aab-bc60-c5c52c76e5db None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:22.622128 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b3500a0f-67dc-4aab-bc60-c5c52c76e5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:22.697230 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1222/2439] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:22.705034 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260534f-efc9-465e-8c3a-a888742da64f None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:22.705326 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260534f-efc9-465e-8c3a-a888742da64f None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:22.705609 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260534f-efc9-465e-8c3a-a888742da64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:22.763491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-0260534f-efc9-465e-8c3a-a888742da64f None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:22.766691 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-0260534f-efc9-465e-8c3a-a888742da64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:22.780314 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1218/2440] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:22.784604 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3500a0f-67dc-4aab-bc60-c5c52c76e5db None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:22.794796 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3500a0f-67dc-4aab-bc60-c5c52c76e5db None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:22.799232 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330d9eb-dd35-43d2-8a23-c9b384ebcd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:22.799910 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330d9eb-dd35-43d2-8a23-c9b384ebcd7a admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:22.800193 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330d9eb-dd35-43d2-8a23-c9b384ebcd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:22.800463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330d9eb-dd35-43d2-8a23-c9b384ebcd7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:22.801181 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330d9eb-dd35-43d2-8a23-c9b384ebcd7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:44:22.801479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330d9eb-dd35-43d2-8a23-c9b384ebcd7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:22.803052 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330d9eb-dd35-43d2-8a23-c9b384ebcd7a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:22.816242 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1223/2441] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:44:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:44:22.820475 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260534f-efc9-465e-8c3a-a888742da64f None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:22.833588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260534f-efc9-465e-8c3a-a888742da64f None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:22.839403 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f15bc-707c-48a1-8563-efa3a08a41c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:22.840108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f15bc-707c-48a1-8563-efa3a08a41c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:22.840465 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f15bc-707c-48a1-8563-efa3a08a41c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:22.840745 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f15bc-707c-48a1-8563-efa3a08a41c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:22.841673 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f15bc-707c-48a1-8563-efa3a08a41c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:22.844253 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f15bc-707c-48a1-8563-efa3a08a41c1 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:22.881956 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1219/2442] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:44:22 2022] POST /identity/v3/projects => generated 363 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:44:22.886937 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330d9eb-dd35-43d2-8a23-c9b384ebcd7a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:22.896921 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330d9eb-dd35-43d2-8a23-c9b384ebcd7a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:22.961336 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d9320d-f702-4b2a-9de2-fcdf6f79c037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:22.962159 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d9320d-f702-4b2a-9de2-fcdf6f79c037 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:22.962521 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d9320d-f702-4b2a-9de2-fcdf6f79c037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:22.962893 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d9320d-f702-4b2a-9de2-fcdf6f79c037 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:22.963600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d9320d-f702-4b2a-9de2-fcdf6f79c037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:22.965831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d9320d-f702-4b2a-9de2-fcdf6f79c037 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:22.984513 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:22.991430 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1224/2443] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:44:22 2022] POST /identity/v3/users => generated 282 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:44:22.995289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f15bc-707c-48a1-8563-efa3a08a41c1 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:23.008622 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f15bc-707c-48a1-8563-efa3a08a41c1 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:23.018521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69715450-b77a-4777-a673-6b0e8adb94b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:23.019488 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69715450-b77a-4777-a673-6b0e8adb94b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:23.019775 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69715450-b77a-4777-a673-6b0e8adb94b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:23.020145 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69715450-b77a-4777-a673-6b0e8adb94b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:23.021005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69715450-b77a-4777-a673-6b0e8adb94b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:23.022889 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69715450-b77a-4777-a673-6b0e8adb94b3 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:23.034427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1220/2444] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:23.038381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d9320d-f702-4b2a-9de2-fcdf6f79c037 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:23.050464 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d9320d-f702-4b2a-9de2-fcdf6f79c037 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:23.055332 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6bd8a-015c-4cfa-971b-6bcbf8776062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:23.056053 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6bd8a-015c-4cfa-971b-6bcbf8776062 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:23.056340 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6bd8a-015c-4cfa-971b-6bcbf8776062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:23.056615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6bd8a-015c-4cfa-971b-6bcbf8776062 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:23.057348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c6bd8a-015c-4cfa-971b-6bcbf8776062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:23.059872 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c6bd8a-015c-4cfa-971b-6bcbf8776062 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:23.072806 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1225/2445] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:23.077314 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69715450-b77a-4777-a673-6b0e8adb94b3 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:23.088333 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69715450-b77a-4777-a673-6b0e8adb94b3 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:23.093540 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e635e1-b953-469a-9117-c367f0c9452d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:23.094413 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e635e1-b953-469a-9117-c367f0c9452d admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:23.094694 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e635e1-b953-469a-9117-c367f0c9452d admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:23.094967 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e635e1-b953-469a-9117-c367f0c9452d admin admin] PATH_INFO: `/v3/projects/32e78e0c79ef4f52845e3701606aaa52/users/0ba7462d762e46d4b02ffd01d95920fe/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:23.110731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e635e1-b953-469a-9117-c367f0c9452d admin admin] RBAC: Authorizing `identity:create_grant(project_id=32e78e0c79ef4f52845e3701606aaa52, user_id=0ba7462d762e46d4b02ffd01d95920fe, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:23.113395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e635e1-b953-469a-9117-c367f0c9452d admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:23.127127 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1221/2446] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:44:23 2022] PUT /identity/v3/projects/32e78e0c79ef4f52845e3701606aaa52/users/0ba7462d762e46d4b02ffd01d95920fe/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:23.139307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b964d-4b12-4209-a3d2-ad1decafed4e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:23.141597 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b964d-4b12-4209-a3d2-ad1decafed4e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:23.141922 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b964d-4b12-4209-a3d2-ad1decafed4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:23.173468 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-4d6b964d-4b12-4209-a3d2-ad1decafed4e None None] MFA Rules not processed for user `0ba7462d762e46d4b02ffd01d95920fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:23.174689 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4d6b964d-4b12-4209-a3d2-ad1decafed4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:23.243016 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1226/2447] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:23 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:23.248182 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e635e1-b953-469a-9117-c367f0c9452d admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:23.323928 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e635e1-b953-469a-9117-c367f0c9452d admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:23.329747 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463a7db-a324-49d3-afd0-1e297553606d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:23.330459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463a7db-a324-49d3-afd0-1e297553606d admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:23.330743 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463a7db-a324-49d3-afd0-1e297553606d admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:23.331022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463a7db-a324-49d3-afd0-1e297553606d admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:23.331888 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463a7db-a324-49d3-afd0-1e297553606d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:23.333853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463a7db-a324-49d3-afd0-1e297553606d admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:23.337365 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:23.369927 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1222/2448] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:44:23 2022] POST /identity/v3/projects => generated 365 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:44:23.374248 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b964d-4b12-4209-a3d2-ad1decafed4e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:23.385977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b964d-4b12-4209-a3d2-ad1decafed4e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:23.448736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25de0ff-fa6c-403c-afc1-33aeb7129790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:23.449982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25de0ff-fa6c-403c-afc1-33aeb7129790 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:23.450480 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25de0ff-fa6c-403c-afc1-33aeb7129790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:23.450909 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25de0ff-fa6c-403c-afc1-33aeb7129790 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:23.451899 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25de0ff-fa6c-403c-afc1-33aeb7129790 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:23.454329 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25de0ff-fa6c-403c-afc1-33aeb7129790 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:23.479822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1227/2449] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:44:23 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:44:23.481783 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463a7db-a324-49d3-afd0-1e297553606d admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:23.492405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463a7db-a324-49d3-afd0-1e297553606d admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:23.497721 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9c01d-abc0-46ef-8fcb-3e718e579035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:23.498385 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9c01d-abc0-46ef-8fcb-3e718e579035 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:23.498673 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9c01d-abc0-46ef-8fcb-3e718e579035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:23.498940 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9c01d-abc0-46ef-8fcb-3e718e579035 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:23.499662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9c01d-abc0-46ef-8fcb-3e718e579035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:23.501350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9c01d-abc0-46ef-8fcb-3e718e579035 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:23.513325 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1223/2450] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:44:23.518058 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25de0ff-fa6c-403c-afc1-33aeb7129790 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:23.529893 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25de0ff-fa6c-403c-afc1-33aeb7129790 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:23.534878 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6471b5-fe94-41ab-950a-453906758a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:23.535871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6471b5-fe94-41ab-950a-453906758a98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:23.536223 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6471b5-fe94-41ab-950a-453906758a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:23.536576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6471b5-fe94-41ab-950a-453906758a98 admin admin] PATH_INFO: `/v3/projects/45fb12a411bf45e7ae26875c2222dfb8/users/95f986f7509541999d330dc89c0d9e6b/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:23.559385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6471b5-fe94-41ab-950a-453906758a98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45fb12a411bf45e7ae26875c2222dfb8, user_id=95f986f7509541999d330dc89c0d9e6b, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:23.559385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6471b5-fe94-41ab-950a-453906758a98 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:23.570119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1228/2451] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:44:23 2022] PUT /identity/v3/projects/45fb12a411bf45e7ae26875c2222dfb8/users/95f986f7509541999d330dc89c0d9e6b/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:23.575954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9c01d-abc0-46ef-8fcb-3e718e579035 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:23.588503 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9c01d-abc0-46ef-8fcb-3e718e579035 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:23.648169 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95989d06-74d9-4aae-a0cd-b0f163b11555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:23.648875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95989d06-74d9-4aae-a0cd-b0f163b11555 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:23.649273 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95989d06-74d9-4aae-a0cd-b0f163b11555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:23.649654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95989d06-74d9-4aae-a0cd-b0f163b11555 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:23.650491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95989d06-74d9-4aae-a0cd-b0f163b11555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:23.652659 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95989d06-74d9-4aae-a0cd-b0f163b11555 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:23.667944 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1224/2452] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:23.670262 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6471b5-fe94-41ab-950a-453906758a98 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:23.686030 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6471b5-fe94-41ab-950a-453906758a98 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:23.706260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20bf29-fc76-48aa-88ca-b7a96213ef17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:23.707327 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20bf29-fc76-48aa-88ca-b7a96213ef17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:23.707713 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20bf29-fc76-48aa-88ca-b7a96213ef17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:23.708108 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20bf29-fc76-48aa-88ca-b7a96213ef17 admin admin] PATH_INFO: `/v3/projects/45fb12a411bf45e7ae26875c2222dfb8/users/95f986f7509541999d330dc89c0d9e6b/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:23.710273 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d20bf29-fc76-48aa-88ca-b7a96213ef17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45fb12a411bf45e7ae26875c2222dfb8, user_id=95f986f7509541999d330dc89c0d9e6b, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:23.713072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d20bf29-fc76-48aa-88ca-b7a96213ef17 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:23.735267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1229/2453] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:44:23 2022] PUT /identity/v3/projects/45fb12a411bf45e7ae26875c2222dfb8/users/95f986f7509541999d330dc89c0d9e6b/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:23.736777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eec36d-44e0-402c-8e20-65467a5b16c5 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:23.737068 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eec36d-44e0-402c-8e20-65467a5b16c5 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:23.737344 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eec36d-44e0-402c-8e20-65467a5b16c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:23.780351 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-c4eec36d-44e0-402c-8e20-65467a5b16c5 None None] MFA Rules not processed for user `95f986f7509541999d330dc89c0d9e6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:23.781659 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-c4eec36d-44e0-402c-8e20-65467a5b16c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:23.916030 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1225/2454] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:23 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:23.929480 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20bf29-fc76-48aa-88ca-b7a96213ef17 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:23.933991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0d20bf29-fc76-48aa-88ca-b7a96213ef17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:23.977881 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:24.049508 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20bf29-fc76-48aa-88ca-b7a96213ef17 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:24.053428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85929d-319c-46f3-a154-da58b5373066 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:24.054106 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85929d-319c-46f3-a154-da58b5373066 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:24.054383 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85929d-319c-46f3-a154-da58b5373066 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:24.054651 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85929d-319c-46f3-a154-da58b5373066 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:24.071435 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a85929d-319c-46f3-a154-da58b5373066 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:24.074331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a85929d-319c-46f3-a154-da58b5373066 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:24.086539 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1230/2455] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:44:23 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:44:24.174468 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eec36d-44e0-402c-8e20-65467a5b16c5 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:24.193220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eec36d-44e0-402c-8e20-65467a5b16c5 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:24.197091 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffca3f-3421-4df8-bc82-60f7e3e393e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:24.197735 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffca3f-3421-4df8-bc82-60f7e3e393e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:24.198015 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffca3f-3421-4df8-bc82-60f7e3e393e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:24.198286 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffca3f-3421-4df8-bc82-60f7e3e393e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:24.259354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:24.265535 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffca3f-3421-4df8-bc82-60f7e3e393e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:24.267401 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffca3f-3421-4df8-bc82-60f7e3e393e0 service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:24.282258 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1226/2456] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:44:24 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:44:33.539409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85929d-319c-46f3-a154-da58b5373066 service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:33.630217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85929d-319c-46f3-a154-da58b5373066 service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:33.635485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ae99e-14f2-4c17-8f1a-5f4b456d58f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:33.637643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ae99e-14f2-4c17-8f1a-5f4b456d58f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:33.637908 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ae99e-14f2-4c17-8f1a-5f4b456d58f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:33.638189 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ae99e-14f2-4c17-8f1a-5f4b456d58f4 admin admin] PATH_INFO: `/v3/users/0ba7462d762e46d4b02ffd01d95920fe` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:33.640384 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ae99e-14f2-4c17-8f1a-5f4b456d58f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ba7462d762e46d4b02ffd01d95920fe)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:33.642786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ae99e-14f2-4c17-8f1a-5f4b456d58f4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:33.693628 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-577ae99e-14f2-4c17-8f1a-5f4b456d58f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ba7462d762e46d4b02ffd01d95920fe'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:33.701011 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-577ae99e-14f2-4c17-8f1a-5f4b456d58f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ba7462d762e46d4b02ffd01d95920fe'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:33.701809 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-577ae99e-14f2-4c17-8f1a-5f4b456d58f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ba7462d762e46d4b02ffd01d95920fe'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:33.702561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-577ae99e-14f2-4c17-8f1a-5f4b456d58f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ba7462d762e46d4b02ffd01d95920fe'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:33.715274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-577ae99e-14f2-4c17-8f1a-5f4b456d58f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ba7462d762e46d4b02ffd01d95920fe'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:33.731525 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1231/2457] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:44:33 2022] DELETE /identity/v3/users/0ba7462d762e46d4b02ffd01d95920fe => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:33.736038 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6686c19-4144-4fe3-96cf-3ec54ae18160 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:33.736568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6686c19-4144-4fe3-96cf-3ec54ae18160 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:33.736568 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6686c19-4144-4fe3-96cf-3ec54ae18160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:33.754078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-d6686c19-4144-4fe3-96cf-3ec54ae18160 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:33.755282 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-d6686c19-4144-4fe3-96cf-3ec54ae18160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:33.824361 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1227/2458] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:33.833871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ae99e-14f2-4c17-8f1a-5f4b456d58f4 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:33.835329 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-577ae99e-14f2-4c17-8f1a-5f4b456d58f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:33.874344 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:33.900087 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ae99e-14f2-4c17-8f1a-5f4b456d58f4 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:33.904697 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc135f5f-c1de-4568-88d9-1d9ead4b3886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:33.905342 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc135f5f-c1de-4568-88d9-1d9ead4b3886 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:33.905625 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc135f5f-c1de-4568-88d9-1d9ead4b3886 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:33.905897 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc135f5f-c1de-4568-88d9-1d9ead4b3886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:33.915135 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc135f5f-c1de-4568-88d9-1d9ead4b3886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:33.916638 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc135f5f-c1de-4568-88d9-1d9ead4b3886 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:33.927418 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1232/2459] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:44:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:44:34.025137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8579b58-b2e1-453b-bda6-014fa4a9b6b9 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:34.025551 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8579b58-b2e1-453b-bda6-014fa4a9b6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:34.026495 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8579b58-b2e1-453b-bda6-014fa4a9b6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:34.051706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-b8579b58-b2e1-453b-bda6-014fa4a9b6b9 None None] MFA Rules not processed for user `e9e358afe0974c0c8bd53104ebbff445`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:34.053580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-b8579b58-b2e1-453b-bda6-014fa4a9b6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:34.122960 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1228/2460] 10.222.0.42 () {58 vars in 967 bytes} [Mon Jun 6 10:44:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:34.139468 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc135f5f-c1de-4568-88d9-1d9ead4b3886 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:34.140945 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bc135f5f-c1de-4568-88d9-1d9ead4b3886 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:34.241468 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc135f5f-c1de-4568-88d9-1d9ead4b3886 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:34.248473 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1274f-6a72-4d3e-8e3d-621f1f867ff8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fede9e52ad274285b09d748a684ede38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:34.249146 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1274f-6a72-4d3e-8e3d-621f1f867ff8 service swift] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:34.249434 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1274f-6a72-4d3e-8e3d-621f1f867ff8 service swift] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:34.249802 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1274f-6a72-4d3e-8e3d-621f1f867ff8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:34.256473 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8579b58-b2e1-453b-bda6-014fa4a9b6b9 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:34.258297 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-b8579b58-b2e1-453b-bda6-014fa4a9b6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:34.260452 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad1274f-6a72-4d3e-8e3d-621f1f867ff8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:34.262960 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad1274f-6a72-4d3e-8e3d-621f1f867ff8 service swift] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:34.274552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1233/2461] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jun 6 10:44:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 10:44:34.285843 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8579b58-b2e1-453b-bda6-014fa4a9b6b9 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:34.290704 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b680905-cd2e-4a0f-b9ed-96ff0541d8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:34.292121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b680905-cd2e-4a0f-b9ed-96ff0541d8df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:34.292503 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b680905-cd2e-4a0f-b9ed-96ff0541d8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:34.292668 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b680905-cd2e-4a0f-b9ed-96ff0541d8df admin admin] PATH_INFO: `/v3/projects/32e78e0c79ef4f52845e3701606aaa52` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:34.293866 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b680905-cd2e-4a0f-b9ed-96ff0541d8df admin admin] RBAC: Authorizing `identity:delete_project(project_id=32e78e0c79ef4f52845e3701606aaa52)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:34.295809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b680905-cd2e-4a0f-b9ed-96ff0541d8df admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:34.355546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2b680905-cd2e-4a0f-b9ed-96ff0541d8df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32e78e0c79ef4f52845e3701606aaa52', 'request_id': 'req-2b680905-cd2e-4a0f-b9ed-96ff0541d8df'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:34.362338 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2b680905-cd2e-4a0f-b9ed-96ff0541d8df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32e78e0c79ef4f52845e3701606aaa52', 'request_id': 'req-2b680905-cd2e-4a0f-b9ed-96ff0541d8df'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:34.372589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2b680905-cd2e-4a0f-b9ed-96ff0541d8df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32e78e0c79ef4f52845e3701606aaa52', 'request_id': 'req-2b680905-cd2e-4a0f-b9ed-96ff0541d8df'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:34.396161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2b680905-cd2e-4a0f-b9ed-96ff0541d8df admin admin] The token cache is being invalidate because project 32e78e0c79ef4f52845e3701606aaa52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:44:34.396554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2b680905-cd2e-4a0f-b9ed-96ff0541d8df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:34.397292 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2b680905-cd2e-4a0f-b9ed-96ff0541d8df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:34.399506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1229/2462] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:44:34 2022] DELETE /identity/v3/projects/32e78e0c79ef4f52845e3701606aaa52 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:44:34.403254 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1274f-6a72-4d3e-8e3d-621f1f867ff8 service swift] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:34.415734 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1274f-6a72-4d3e-8e3d-621f1f867ff8 service swift] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:34.500979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd1f4b-45d0-4e49-9a5c-8863f5fed09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:34.502942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd1f4b-45d0-4e49-9a5c-8863f5fed09b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:34.503572 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd1f4b-45d0-4e49-9a5c-8863f5fed09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:34.504331 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd1f4b-45d0-4e49-9a5c-8863f5fed09b admin admin] PATH_INFO: `/v3/users/95f986f7509541999d330dc89c0d9e6b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:34.504460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fdbbb-43e9-47b3-bfdb-d9c1f38aec51 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:34.504631 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fdbbb-43e9-47b3-bfdb-d9c1f38aec51 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:34.504933 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fdbbb-43e9-47b3-bfdb-d9c1f38aec51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:34.507655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd1f4b-45d0-4e49-9a5c-8863f5fed09b admin admin] RBAC: Authorizing `identity:delete_user(user_id=95f986f7509541999d330dc89c0d9e6b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:34.511602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd1f4b-45d0-4e49-9a5c-8863f5fed09b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:34.551286 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-416fdbbb-43e9-47b3-bfdb-d9c1f38aec51 None None] MFA Rules not processed for user `7ba341ea1ac34874bbfb8b2f713aea1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:34.553225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-416fdbbb-43e9-47b3-bfdb-d9c1f38aec51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:34.577221 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3ecd1f4b-45d0-4e49-9a5c-8863f5fed09b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95f986f7509541999d330dc89c0d9e6b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:34.578425 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:34.586548 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3ecd1f4b-45d0-4e49-9a5c-8863f5fed09b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95f986f7509541999d330dc89c0d9e6b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:34.587657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3ecd1f4b-45d0-4e49-9a5c-8863f5fed09b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95f986f7509541999d330dc89c0d9e6b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:34.588729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3ecd1f4b-45d0-4e49-9a5c-8863f5fed09b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95f986f7509541999d330dc89c0d9e6b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:34.600480 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:34.608992 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3ecd1f4b-45d0-4e49-9a5c-8863f5fed09b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95f986f7509541999d330dc89c0d9e6b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:34.631390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1234/2463] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:44:34 2022] DELETE /identity/v3/users/95f986f7509541999d330dc89c0d9e6b => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:34.658091 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1230/2464] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:44:34.964477 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd1f4b-45d0-4e49-9a5c-8863f5fed09b admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:34.966170 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3ecd1f4b-45d0-4e49-9a5c-8863f5fed09b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:35.060788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd1f4b-45d0-4e49-9a5c-8863f5fed09b admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:35.068950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfcc982-ee81-401b-b7b8-aec2d28a4153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:35.069845 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfcc982-ee81-401b-b7b8-aec2d28a4153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:35.070131 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfcc982-ee81-401b-b7b8-aec2d28a4153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:35.070409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfcc982-ee81-401b-b7b8-aec2d28a4153 admin admin] PATH_INFO: `/v3/projects/45fb12a411bf45e7ae26875c2222dfb8` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:35.071675 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfcc982-ee81-401b-b7b8-aec2d28a4153 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45fb12a411bf45e7ae26875c2222dfb8)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:35.073750 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfcc982-ee81-401b-b7b8-aec2d28a4153 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:35.121884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ebfcc982-ee81-401b-b7b8-aec2d28a4153 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45fb12a411bf45e7ae26875c2222dfb8', 'request_id': 'req-ebfcc982-ee81-401b-b7b8-aec2d28a4153'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:35.129668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ebfcc982-ee81-401b-b7b8-aec2d28a4153 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45fb12a411bf45e7ae26875c2222dfb8', 'request_id': 'req-ebfcc982-ee81-401b-b7b8-aec2d28a4153'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:35.135112 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ebfcc982-ee81-401b-b7b8-aec2d28a4153 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45fb12a411bf45e7ae26875c2222dfb8', 'request_id': 'req-ebfcc982-ee81-401b-b7b8-aec2d28a4153'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:35.147101 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ebfcc982-ee81-401b-b7b8-aec2d28a4153 admin admin] The token cache is being invalidate because project 45fb12a411bf45e7ae26875c2222dfb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:44:35.147332 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ebfcc982-ee81-401b-b7b8-aec2d28a4153 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:35.147990 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ebfcc982-ee81-401b-b7b8-aec2d28a4153 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:35.149539 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1235/2465] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:44:34 2022] DELETE /identity/v3/projects/45fb12a411bf45e7ae26875c2222dfb8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:35.156599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4d529-d5ff-4172-ba61-28bed9220c79 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:35.156599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4d529-d5ff-4172-ba61-28bed9220c79 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:35.156599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4d529-d5ff-4172-ba61-28bed9220c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:35.173993 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-24d4d529-d5ff-4172-ba61-28bed9220c79 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:35.175624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-24d4d529-d5ff-4172-ba61-28bed9220c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:35.175889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfcc982-ee81-401b-b7b8-aec2d28a4153 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:35.177254 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ebfcc982-ee81-401b-b7b8-aec2d28a4153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:35.234665 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfcc982-ee81-401b-b7b8-aec2d28a4153 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:35.238281 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1231/2466] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:35.238881 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e715c2-db85-4a76-a815-0302c46ffe4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:35.239576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e715c2-db85-4a76-a815-0302c46ffe4b service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:35.239899 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e715c2-db85-4a76-a815-0302c46ffe4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:35.240176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e715c2-db85-4a76-a815-0302c46ffe4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:35.242126 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d6e715c2-db85-4a76-a815-0302c46ffe4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:35.246819 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8004524-eec6-4619-afaf-6bd0446021e0 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:35.247115 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8004524-eec6-4619-afaf-6bd0446021e0 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:35.247395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8004524-eec6-4619-afaf-6bd0446021e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:35.253363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e715c2-db85-4a76-a815-0302c46ffe4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:35.255797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e715c2-db85-4a76-a815-0302c46ffe4b service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:35.265073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1236/2467] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:44:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:44:35.265850 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-a8004524-eec6-4619-afaf-6bd0446021e0 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:35.267072 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a8004524-eec6-4619-afaf-6bd0446021e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:35.276875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1232/2468] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:35.281034 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e715c2-db85-4a76-a815-0302c46ffe4b service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:35.292720 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e715c2-db85-4a76-a815-0302c46ffe4b service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:35.297701 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77d674-72fa-49a3-a7d4-caaa1e86442a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:35.298353 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77d674-72fa-49a3-a7d4-caaa1e86442a admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:35.298641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77d674-72fa-49a3-a7d4-caaa1e86442a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:35.298917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77d674-72fa-49a3-a7d4-caaa1e86442a admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:35.299643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77d674-72fa-49a3-a7d4-caaa1e86442a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:44:35.299985 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77d674-72fa-49a3-a7d4-caaa1e86442a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:35.301615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77d674-72fa-49a3-a7d4-caaa1e86442a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:35.315247 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1237/2469] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:44:35.320837 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8004524-eec6-4619-afaf-6bd0446021e0 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:35.330932 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8004524-eec6-4619-afaf-6bd0446021e0 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:35.335768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d38d95-0cd4-4520-b9a2-da9fbba40e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:35.336451 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d38d95-0cd4-4520-b9a2-da9fbba40e7f admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:35.336736 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d38d95-0cd4-4520-b9a2-da9fbba40e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:35.337672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d38d95-0cd4-4520-b9a2-da9fbba40e7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:35.338919 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d38d95-0cd4-4520-b9a2-da9fbba40e7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:35.340885 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d38d95-0cd4-4520-b9a2-da9fbba40e7f admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:35.367334 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1233/2470] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:44:35 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:44:35.371808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77d674-72fa-49a3-a7d4-caaa1e86442a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:35.378983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:35.383301 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77d674-72fa-49a3-a7d4-caaa1e86442a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:35.445885 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe7e9a-6732-487b-be94-22a4ed90cfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:35.446567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe7e9a-6732-487b-be94-22a4ed90cfaa admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:35.446905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe7e9a-6732-487b-be94-22a4ed90cfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:35.447191 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe7e9a-6732-487b-be94-22a4ed90cfaa admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:35.448000 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe7e9a-6732-487b-be94-22a4ed90cfaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:35.450135 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe7e9a-6732-487b-be94-22a4ed90cfaa admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:35.472685 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1238/2471] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:44:35 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:44:35.476786 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d38d95-0cd4-4520-b9a2-da9fbba40e7f admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:35.487620 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d38d95-0cd4-4520-b9a2-da9fbba40e7f admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:35.492609 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c43b7-c988-42f9-a681-c59442752eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:35.493261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c43b7-c988-42f9-a681-c59442752eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:35.493543 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c43b7-c988-42f9-a681-c59442752eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:35.493818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c43b7-c988-42f9-a681-c59442752eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:35.494559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c43b7-c988-42f9-a681-c59442752eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:35.496197 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c43b7-c988-42f9-a681-c59442752eb6 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:35.506721 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1234/2472] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:35.510704 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe7e9a-6732-487b-be94-22a4ed90cfaa admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:35.521725 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe7e9a-6732-487b-be94-22a4ed90cfaa admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:35.527277 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466933b-0133-4272-ac32-c2dd777fefc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:35.528051 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466933b-0133-4272-ac32-c2dd777fefc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:35.528508 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466933b-0133-4272-ac32-c2dd777fefc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:35.528824 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466933b-0133-4272-ac32-c2dd777fefc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:35.529867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466933b-0133-4272-ac32-c2dd777fefc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:35.531487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466933b-0133-4272-ac32-c2dd777fefc3 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:35.543484 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1239/2473] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:35.547905 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c43b7-c988-42f9-a681-c59442752eb6 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:35.563734 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c43b7-c988-42f9-a681-c59442752eb6 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:35.580311 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289bdfa5-b227-4759-b3b3-30f7f41c4d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:35.581224 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bdfa5-b227-4759-b3b3-30f7f41c4d98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:35.581503 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bdfa5-b227-4759-b3b3-30f7f41c4d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:35.581791 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bdfa5-b227-4759-b3b3-30f7f41c4d98 admin admin] PATH_INFO: `/v3/projects/320d29f4a637485596ec502f0defdcf6/users/3b5fe880a3b74a47b2edd459796d1726/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:35.601949 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289bdfa5-b227-4759-b3b3-30f7f41c4d98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=320d29f4a637485596ec502f0defdcf6, user_id=3b5fe880a3b74a47b2edd459796d1726, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:35.606206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289bdfa5-b227-4759-b3b3-30f7f41c4d98 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:35.631501 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1235/2474] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:44:35 2022] PUT /identity/v3/projects/320d29f4a637485596ec502f0defdcf6/users/3b5fe880a3b74a47b2edd459796d1726/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:35.634595 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc420959-7439-4a77-bc2a-b2fcdd849d75 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:35.635049 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc420959-7439-4a77-bc2a-b2fcdd849d75 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:35.636073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc420959-7439-4a77-bc2a-b2fcdd849d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:35.666047 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-bc420959-7439-4a77-bc2a-b2fcdd849d75 None None] MFA Rules not processed for user `3b5fe880a3b74a47b2edd459796d1726`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:35.668024 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bc420959-7439-4a77-bc2a-b2fcdd849d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:35.733772 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1240/2475] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:35 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:35.738734 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289bdfa5-b227-4759-b3b3-30f7f41c4d98 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:35.806812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289bdfa5-b227-4759-b3b3-30f7f41c4d98 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:35.812522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca750ea-d463-4a20-bf81-fad835985998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:35.813238 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca750ea-d463-4a20-bf81-fad835985998 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:35.813459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca750ea-d463-4a20-bf81-fad835985998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:35.813732 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca750ea-d463-4a20-bf81-fad835985998 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:35.814423 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca750ea-d463-4a20-bf81-fad835985998 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:35.816370 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca750ea-d463-4a20-bf81-fad835985998 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:35.825024 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:35.846203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1236/2476] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:44:35 2022] POST /identity/v3/projects => generated 369 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:44:35.850101 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc420959-7439-4a77-bc2a-b2fcdd849d75 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:35.861467 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc420959-7439-4a77-bc2a-b2fcdd849d75 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:35.928140 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c8cc8-f5f6-4ed9-bc94-d3354a3ecbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:35.928844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c8cc8-f5f6-4ed9-bc94-d3354a3ecbb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:35.929121 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c8cc8-f5f6-4ed9-bc94-d3354a3ecbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:35.929390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c8cc8-f5f6-4ed9-bc94-d3354a3ecbb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:35.930137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c8cc8-f5f6-4ed9-bc94-d3354a3ecbb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:35.932754 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c8cc8-f5f6-4ed9-bc94-d3354a3ecbb4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:35.955968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1241/2477] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:44:35 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:44:35.957230 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca750ea-d463-4a20-bf81-fad835985998 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:35.968191 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca750ea-d463-4a20-bf81-fad835985998 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:35.972904 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06bc0a-3a56-4f7f-8599-0c61b936ad47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:35.973567 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06bc0a-3a56-4f7f-8599-0c61b936ad47 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:35.973852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06bc0a-3a56-4f7f-8599-0c61b936ad47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:35.974127 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06bc0a-3a56-4f7f-8599-0c61b936ad47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:35.974842 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06bc0a-3a56-4f7f-8599-0c61b936ad47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:35.976483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06bc0a-3a56-4f7f-8599-0c61b936ad47 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:35.988196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1237/2478] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:35.993392 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c8cc8-f5f6-4ed9-bc94-d3354a3ecbb4 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:36.000097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:36.006921 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c8cc8-f5f6-4ed9-bc94-d3354a3ecbb4 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:36.012063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f4d53-f778-4e63-82a1-30ff7f48dea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:36.013031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f4d53-f778-4e63-82a1-30ff7f48dea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:36.013393 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f4d53-f778-4e63-82a1-30ff7f48dea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:36.013703 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f4d53-f778-4e63-82a1-30ff7f48dea5 admin admin] PATH_INFO: `/v3/projects/8b7996dc47ff436e91aba79e95f63d10/users/d896a5f2e61547f1b01109f51ac74b38/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:36.030921 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f4d53-f778-4e63-82a1-30ff7f48dea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b7996dc47ff436e91aba79e95f63d10, user_id=d896a5f2e61547f1b01109f51ac74b38, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:36.033647 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f4d53-f778-4e63-82a1-30ff7f48dea5 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:36.049084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1242/2479] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:44:35 2022] PUT /identity/v3/projects/8b7996dc47ff436e91aba79e95f63d10/users/d896a5f2e61547f1b01109f51ac74b38/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:44:36.052910 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06bc0a-3a56-4f7f-8599-0c61b936ad47 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:36.067967 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06bc0a-3a56-4f7f-8599-0c61b936ad47 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:36.141894 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7548eab0-8f85-43cb-af01-af152402a65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:36.142556 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548eab0-8f85-43cb-af01-af152402a65a admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:36.142837 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548eab0-8f85-43cb-af01-af152402a65a admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:36.143111 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548eab0-8f85-43cb-af01-af152402a65a admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:36.143864 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548eab0-8f85-43cb-af01-af152402a65a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:36.145457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548eab0-8f85-43cb-af01-af152402a65a admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:36.161708 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1238/2480] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:36.165981 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f4d53-f778-4e63-82a1-30ff7f48dea5 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:36.177965 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f4d53-f778-4e63-82a1-30ff7f48dea5 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:36.182819 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d13b89-cd9e-40d6-8065-e10c1fdeccd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:36.183885 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d13b89-cd9e-40d6-8065-e10c1fdeccd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:36.184165 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d13b89-cd9e-40d6-8065-e10c1fdeccd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:36.184434 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d13b89-cd9e-40d6-8065-e10c1fdeccd7 admin admin] PATH_INFO: `/v3/projects/8b7996dc47ff436e91aba79e95f63d10/users/d896a5f2e61547f1b01109f51ac74b38/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:36.186387 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d13b89-cd9e-40d6-8065-e10c1fdeccd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b7996dc47ff436e91aba79e95f63d10, user_id=d896a5f2e61547f1b01109f51ac74b38, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:36.189047 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d13b89-cd9e-40d6-8065-e10c1fdeccd7 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:36.200523 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1243/2481] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:44:36 2022] PUT /identity/v3/projects/8b7996dc47ff436e91aba79e95f63d10/users/d896a5f2e61547f1b01109f51ac74b38/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:44:36.211227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98127b-c000-4365-ab50-bb7d4bfd9e0b None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:36.211227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98127b-c000-4365-ab50-bb7d4bfd9e0b None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:36.211486 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98127b-c000-4365-ab50-bb7d4bfd9e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:36.244039 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-0c98127b-c000-4365-ab50-bb7d4bfd9e0b None None] MFA Rules not processed for user `d896a5f2e61547f1b01109f51ac74b38`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:36.244039 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-0c98127b-c000-4365-ab50-bb7d4bfd9e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:36.309763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1239/2482] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:36 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:36.322467 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d13b89-cd9e-40d6-8065-e10c1fdeccd7 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:36.324383 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-40d13b89-cd9e-40d6-8065-e10c1fdeccd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:36.388506 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d13b89-cd9e-40d6-8065-e10c1fdeccd7 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:36.392655 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae64cb7-e463-4bab-932c-fb48e9617a5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:36.393493 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae64cb7-e463-4bab-932c-fb48e9617a5b service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:36.393852 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae64cb7-e463-4bab-932c-fb48e9617a5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:36.394227 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae64cb7-e463-4bab-932c-fb48e9617a5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:36.404564 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae64cb7-e463-4bab-932c-fb48e9617a5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:36.407390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae64cb7-e463-4bab-932c-fb48e9617a5b service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:36.417466 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1244/2483] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:44:36 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:44:37.840171 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98127b-c000-4365-ab50-bb7d4bfd9e0b None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:37.918335 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98127b-c000-4365-ab50-bb7d4bfd9e0b None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:37.925308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468df5a2-c8b6-47fe-9ef6-f6b265d8bc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:37.926558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468df5a2-c8b6-47fe-9ef6-f6b265d8bc7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:37.926846 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468df5a2-c8b6-47fe-9ef6-f6b265d8bc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:37.927126 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468df5a2-c8b6-47fe-9ef6-f6b265d8bc7a admin admin] PATH_INFO: `/v3/users/3b5fe880a3b74a47b2edd459796d1726` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:37.928456 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468df5a2-c8b6-47fe-9ef6-f6b265d8bc7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b5fe880a3b74a47b2edd459796d1726)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:37.930462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468df5a2-c8b6-47fe-9ef6-f6b265d8bc7a admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:37.932443 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:37.989385 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-468df5a2-c8b6-47fe-9ef6-f6b265d8bc7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b5fe880a3b74a47b2edd459796d1726'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:37.996771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-468df5a2-c8b6-47fe-9ef6-f6b265d8bc7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b5fe880a3b74a47b2edd459796d1726'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:38.005647 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-468df5a2-c8b6-47fe-9ef6-f6b265d8bc7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b5fe880a3b74a47b2edd459796d1726'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:38.006271 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-468df5a2-c8b6-47fe-9ef6-f6b265d8bc7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b5fe880a3b74a47b2edd459796d1726'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:38.021190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-468df5a2-c8b6-47fe-9ef6-f6b265d8bc7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b5fe880a3b74a47b2edd459796d1726'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:38.030615 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1240/2484] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:44:37 2022] DELETE /identity/v3/users/3b5fe880a3b74a47b2edd459796d1726 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:38.033255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0046f2-70db-4bf7-94a2-3d75e94692c3 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:38.034122 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0046f2-70db-4bf7-94a2-3d75e94692c3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:38.034560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0046f2-70db-4bf7-94a2-3d75e94692c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:38.053961 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-af0046f2-70db-4bf7-94a2-3d75e94692c3 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:38.055114 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-af0046f2-70db-4bf7-94a2-3d75e94692c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:38.120951 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1245/2485] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:38.131666 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468df5a2-c8b6-47fe-9ef6-f6b265d8bc7a admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:38.133235 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-468df5a2-c8b6-47fe-9ef6-f6b265d8bc7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:38.233384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468df5a2-c8b6-47fe-9ef6-f6b265d8bc7a admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:38.237188 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8502c-b654-44b9-97c8-8dbcc1756217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:38.237835 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8502c-b654-44b9-97c8-8dbcc1756217 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:38.238127 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8502c-b654-44b9-97c8-8dbcc1756217 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:38.238390 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8502c-b654-44b9-97c8-8dbcc1756217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:38.249261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e8502c-b654-44b9-97c8-8dbcc1756217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:38.250850 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e8502c-b654-44b9-97c8-8dbcc1756217 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:38.262219 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1241/2486] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:44:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:44:38.542768 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0046f2-70db-4bf7-94a2-3d75e94692c3 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:38.544239 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-af0046f2-70db-4bf7-94a2-3d75e94692c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:38.557445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0046f2-70db-4bf7-94a2-3d75e94692c3 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:38.562793 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557903c-f6b2-4ff8-9edf-c4adc7f0e654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:38.564182 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557903c-f6b2-4ff8-9edf-c4adc7f0e654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:38.564182 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557903c-f6b2-4ff8-9edf-c4adc7f0e654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:38.564182 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557903c-f6b2-4ff8-9edf-c4adc7f0e654 admin admin] PATH_INFO: `/v3/projects/320d29f4a637485596ec502f0defdcf6` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:38.565561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557903c-f6b2-4ff8-9edf-c4adc7f0e654 admin admin] RBAC: Authorizing `identity:delete_project(project_id=320d29f4a637485596ec502f0defdcf6)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:38.567507 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557903c-f6b2-4ff8-9edf-c4adc7f0e654 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:38.620637 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d557903c-f6b2-4ff8-9edf-c4adc7f0e654 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '320d29f4a637485596ec502f0defdcf6', 'request_id': 'req-d557903c-f6b2-4ff8-9edf-c4adc7f0e654'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:38.627863 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d557903c-f6b2-4ff8-9edf-c4adc7f0e654 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '320d29f4a637485596ec502f0defdcf6', 'request_id': 'req-d557903c-f6b2-4ff8-9edf-c4adc7f0e654'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:38.632807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:38.634025 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d557903c-f6b2-4ff8-9edf-c4adc7f0e654 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '320d29f4a637485596ec502f0defdcf6', 'request_id': 'req-d557903c-f6b2-4ff8-9edf-c4adc7f0e654'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:38.647608 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d557903c-f6b2-4ff8-9edf-c4adc7f0e654 admin admin] The token cache is being invalidate because project 320d29f4a637485596ec502f0defdcf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:44:38.647924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d557903c-f6b2-4ff8-9edf-c4adc7f0e654 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:38.648564 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d557903c-f6b2-4ff8-9edf-c4adc7f0e654 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:38.650104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1246/2487] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:44:38 2022] DELETE /identity/v3/projects/320d29f4a637485596ec502f0defdcf6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:38.654283 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8502c-b654-44b9-97c8-8dbcc1756217 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:38.663894 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8502c-b654-44b9-97c8-8dbcc1756217 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:38.723472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607e1d1-78fb-4307-82f7-c7ee9227a774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:38.724493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607e1d1-78fb-4307-82f7-c7ee9227a774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:38.724778 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607e1d1-78fb-4307-82f7-c7ee9227a774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:38.725054 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607e1d1-78fb-4307-82f7-c7ee9227a774 admin admin] PATH_INFO: `/v3/users/d896a5f2e61547f1b01109f51ac74b38` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:38.726236 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607e1d1-78fb-4307-82f7-c7ee9227a774 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d896a5f2e61547f1b01109f51ac74b38)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:38.728317 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607e1d1-78fb-4307-82f7-c7ee9227a774 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:38.751515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:38.776646 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7607e1d1-78fb-4307-82f7-c7ee9227a774 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd896a5f2e61547f1b01109f51ac74b38'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:38.783546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7607e1d1-78fb-4307-82f7-c7ee9227a774 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd896a5f2e61547f1b01109f51ac74b38'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:38.790538 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7607e1d1-78fb-4307-82f7-c7ee9227a774 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd896a5f2e61547f1b01109f51ac74b38'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:38.791185 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7607e1d1-78fb-4307-82f7-c7ee9227a774 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd896a5f2e61547f1b01109f51ac74b38'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:38.803472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7607e1d1-78fb-4307-82f7-c7ee9227a774 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd896a5f2e61547f1b01109f51ac74b38'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:38.810108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1242/2488] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:44:38 2022] DELETE /identity/v3/users/d896a5f2e61547f1b01109f51ac74b38 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:39.092576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557903c-f6b2-4ff8-9edf-c4adc7f0e654 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:39.094238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d557903c-f6b2-4ff8-9edf-c4adc7f0e654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:39.122950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557903c-f6b2-4ff8-9edf-c4adc7f0e654 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:39.181636 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c65ad-4aa7-4454-8f62-a74c60bbb131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:39.182635 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c65ad-4aa7-4454-8f62-a74c60bbb131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:39.182920 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c65ad-4aa7-4454-8f62-a74c60bbb131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:39.183300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c65ad-4aa7-4454-8f62-a74c60bbb131 admin admin] PATH_INFO: `/v3/projects/8b7996dc47ff436e91aba79e95f63d10` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:39.184503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698c65ad-4aa7-4454-8f62-a74c60bbb131 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b7996dc47ff436e91aba79e95f63d10)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:39.186447 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698c65ad-4aa7-4454-8f62-a74c60bbb131 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:39.243029 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-698c65ad-4aa7-4454-8f62-a74c60bbb131 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b7996dc47ff436e91aba79e95f63d10', 'request_id': 'req-698c65ad-4aa7-4454-8f62-a74c60bbb131'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:39.250485 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-698c65ad-4aa7-4454-8f62-a74c60bbb131 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b7996dc47ff436e91aba79e95f63d10', 'request_id': 'req-698c65ad-4aa7-4454-8f62-a74c60bbb131'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:39.257630 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-698c65ad-4aa7-4454-8f62-a74c60bbb131 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b7996dc47ff436e91aba79e95f63d10', 'request_id': 'req-698c65ad-4aa7-4454-8f62-a74c60bbb131'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:39.269221 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-698c65ad-4aa7-4454-8f62-a74c60bbb131 admin admin] The token cache is being invalidate because project 8b7996dc47ff436e91aba79e95f63d10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:44:39.269446 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-698c65ad-4aa7-4454-8f62-a74c60bbb131 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:39.270225 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-698c65ad-4aa7-4454-8f62-a74c60bbb131 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:39.271775 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1247/2489] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:44:39 2022] DELETE /identity/v3/projects/8b7996dc47ff436e91aba79e95f63d10 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:39.277621 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195be205-fa03-462d-acc4-f5663888fe70 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:39.278087 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195be205-fa03-462d-acc4-f5663888fe70 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:39.278182 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195be205-fa03-462d-acc4-f5663888fe70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:39.297073 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-195be205-fa03-462d-acc4-f5663888fe70 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:39.298276 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-195be205-fa03-462d-acc4-f5663888fe70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:39.361971 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1243/2490] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:39.371106 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc3a6d-817c-4c45-a26f-1402cab8f2a8 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:39.371492 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc3a6d-817c-4c45-a26f-1402cab8f2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:39.372065 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc3a6d-817c-4c45-a26f-1402cab8f2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:39.393781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-1dfc3a6d-817c-4c45-a26f-1402cab8f2a8 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:39.395210 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1dfc3a6d-817c-4c45-a26f-1402cab8f2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:39.403985 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1248/2491] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:39.408271 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195be205-fa03-462d-acc4-f5663888fe70 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:39.418898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195be205-fa03-462d-acc4-f5663888fe70 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:39.423755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c4c2e-c28c-4d96-a2e8-5326d10ff7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:39.424440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c4c2e-c28c-4d96-a2e8-5326d10ff7ec admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:39.424724 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c4c2e-c28c-4d96-a2e8-5326d10ff7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:39.425000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c4c2e-c28c-4d96-a2e8-5326d10ff7ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:39.425736 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59c4c2e-c28c-4d96-a2e8-5326d10ff7ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:44:39.426046 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59c4c2e-c28c-4d96-a2e8-5326d10ff7ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:39.427677 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59c4c2e-c28c-4d96-a2e8-5326d10ff7ec admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:39.442825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1244/2492] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:44:39.444086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc3a6d-817c-4c45-a26f-1402cab8f2a8 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:39.454028 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc3a6d-817c-4c45-a26f-1402cab8f2a8 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:39.458770 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427f0f0-8558-4b34-9bdf-b3f645c68ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:39.459417 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427f0f0-8558-4b34-9bdf-b3f645c68ce2 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:39.459700 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427f0f0-8558-4b34-9bdf-b3f645c68ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:39.460004 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427f0f0-8558-4b34-9bdf-b3f645c68ce2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:39.460683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6427f0f0-8558-4b34-9bdf-b3f645c68ce2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:39.462557 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6427f0f0-8558-4b34-9bdf-b3f645c68ce2 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:39.489083 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1249/2493] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:44:39 2022] POST /identity/v3/projects => generated 359 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:44:39.493337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c4c2e-c28c-4d96-a2e8-5326d10ff7ec admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:39.508880 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c4c2e-c28c-4d96-a2e8-5326d10ff7ec admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:39.563481 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d8c67-b466-477c-a22c-fce98445aea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:39.564180 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d8c67-b466-477c-a22c-fce98445aea9 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:39.564462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d8c67-b466-477c-a22c-fce98445aea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:39.564736 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d8c67-b466-477c-a22c-fce98445aea9 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:39.565428 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d8c67-b466-477c-a22c-fce98445aea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:39.567342 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d8c67-b466-477c-a22c-fce98445aea9 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:39.588121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1245/2494] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:44:39 2022] POST /identity/v3/users => generated 280 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:44:39.592938 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427f0f0-8558-4b34-9bdf-b3f645c68ce2 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:39.603325 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427f0f0-8558-4b34-9bdf-b3f645c68ce2 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:39.608560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803df7b-c917-4b5e-aba0-6d78360b3edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:39.609220 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803df7b-c917-4b5e-aba0-6d78360b3edb admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:39.609503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803df7b-c917-4b5e-aba0-6d78360b3edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:39.609779 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803df7b-c917-4b5e-aba0-6d78360b3edb admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:39.610503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803df7b-c917-4b5e-aba0-6d78360b3edb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:39.612757 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803df7b-c917-4b5e-aba0-6d78360b3edb admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:39.623478 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1250/2495] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:39.627521 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d8c67-b466-477c-a22c-fce98445aea9 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:39.638218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d8c67-b466-477c-a22c-fce98445aea9 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:39.643402 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7a3c2-43f6-4272-aa22-f3cd1444f3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:39.644133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7a3c2-43f6-4272-aa22-f3cd1444f3fc admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:39.644424 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7a3c2-43f6-4272-aa22-f3cd1444f3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:39.644700 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7a3c2-43f6-4272-aa22-f3cd1444f3fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:39.645422 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7a3c2-43f6-4272-aa22-f3cd1444f3fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:39.647026 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7a3c2-43f6-4272-aa22-f3cd1444f3fc admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:39.657840 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1246/2496] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:39.661958 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803df7b-c917-4b5e-aba0-6d78360b3edb admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:39.672297 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803df7b-c917-4b5e-aba0-6d78360b3edb admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:39.677146 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116b2b6-c0d2-43c8-9a9b-fd8887e5fb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:39.678017 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116b2b6-c0d2-43c8-9a9b-fd8887e5fb27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:39.678296 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116b2b6-c0d2-43c8-9a9b-fd8887e5fb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:39.678569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116b2b6-c0d2-43c8-9a9b-fd8887e5fb27 admin admin] PATH_INFO: `/v3/projects/de48895a84ba4614997d91e7881876ae/users/60cb5b2b03f94b8c9fc94909eb50f0e7/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:39.694469 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116b2b6-c0d2-43c8-9a9b-fd8887e5fb27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de48895a84ba4614997d91e7881876ae, user_id=60cb5b2b03f94b8c9fc94909eb50f0e7, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:39.697134 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116b2b6-c0d2-43c8-9a9b-fd8887e5fb27 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:39.711514 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1251/2497] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:44:39 2022] PUT /identity/v3/projects/de48895a84ba4614997d91e7881876ae/users/60cb5b2b03f94b8c9fc94909eb50f0e7/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:39.722517 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b9c82-7904-4caf-8294-b1e8829862c2 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:39.722812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b9c82-7904-4caf-8294-b1e8829862c2 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:39.723094 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b9c82-7904-4caf-8294-b1e8829862c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:39.735646 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:39.751603 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-654b9c82-7904-4caf-8294-b1e8829862c2 None None] MFA Rules not processed for user `60cb5b2b03f94b8c9fc94909eb50f0e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:39.752910 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-654b9c82-7904-4caf-8294-b1e8829862c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:39.820572 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1247/2498] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:39 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:44:39.851376 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116b2b6-c0d2-43c8-9a9b-fd8887e5fb27 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:39.853136 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1116b2b6-c0d2-43c8-9a9b-fd8887e5fb27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:39.874742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:39.913737 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116b2b6-c0d2-43c8-9a9b-fd8887e5fb27 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:39.917693 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba2339-1fac-4a00-8d3c-1ab6ef21f14d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:39.918437 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba2339-1fac-4a00-8d3c-1ab6ef21f14d service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:39.918724 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba2339-1fac-4a00-8d3c-1ab6ef21f14d service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:39.919202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba2339-1fac-4a00-8d3c-1ab6ef21f14d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:39.930317 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba2339-1fac-4a00-8d3c-1ab6ef21f14d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:39.932294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba2339-1fac-4a00-8d3c-1ab6ef21f14d service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:39.941066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1252/2499] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:44:39 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:44:39.955913 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b9c82-7904-4caf-8294-b1e8829862c2 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:40.019106 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b9c82-7904-4caf-8294-b1e8829862c2 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:40.023647 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8497f-e6b9-4879-b32f-5c0ff98deadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:40.024536 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8497f-e6b9-4879-b32f-5c0ff98deadb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:40.024821 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8497f-e6b9-4879-b32f-5c0ff98deadb admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:40.025097 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8497f-e6b9-4879-b32f-5c0ff98deadb admin admin] PATH_INFO: `/v3/users/60cb5b2b03f94b8c9fc94909eb50f0e7` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:40.026537 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8497f-e6b9-4879-b32f-5c0ff98deadb admin admin] RBAC: Authorizing `identity:delete_user(user_id=60cb5b2b03f94b8c9fc94909eb50f0e7)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:40.028491 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8497f-e6b9-4879-b32f-5c0ff98deadb admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:40.075423 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2ac8497f-e6b9-4879-b32f-5c0ff98deadb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60cb5b2b03f94b8c9fc94909eb50f0e7'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:40.082234 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2ac8497f-e6b9-4879-b32f-5c0ff98deadb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60cb5b2b03f94b8c9fc94909eb50f0e7'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:40.090354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2ac8497f-e6b9-4879-b32f-5c0ff98deadb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60cb5b2b03f94b8c9fc94909eb50f0e7'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:40.091148 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2ac8497f-e6b9-4879-b32f-5c0ff98deadb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60cb5b2b03f94b8c9fc94909eb50f0e7'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:40.103734 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-2ac8497f-e6b9-4879-b32f-5c0ff98deadb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60cb5b2b03f94b8c9fc94909eb50f0e7'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:40.110592 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1248/2500] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:44:39 2022] DELETE /identity/v3/users/60cb5b2b03f94b8c9fc94909eb50f0e7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:40.115204 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51f1b6-192e-48e3-864f-4900f07e92f7 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:40.115665 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51f1b6-192e-48e3-864f-4900f07e92f7 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:40.115891 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51f1b6-192e-48e3-864f-4900f07e92f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:40.133893 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ef51f1b6-192e-48e3-864f-4900f07e92f7 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:40.136127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ef51f1b6-192e-48e3-864f-4900f07e92f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:40.194942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1253/2501] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:40.204789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8497f-e6b9-4879-b32f-5c0ff98deadb admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:40.206181 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-2ac8497f-e6b9-4879-b32f-5c0ff98deadb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:40.224662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:40.267284 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8497f-e6b9-4879-b32f-5c0ff98deadb admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:40.270948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82222863-1cc8-4faf-8f6d-bad07c8391da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:40.271588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82222863-1cc8-4faf-8f6d-bad07c8391da service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:40.271930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82222863-1cc8-4faf-8f6d-bad07c8391da service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:40.272205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82222863-1cc8-4faf-8f6d-bad07c8391da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:40.282404 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82222863-1cc8-4faf-8f6d-bad07c8391da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:40.284949 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82222863-1cc8-4faf-8f6d-bad07c8391da service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:40.296293 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1249/2502] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:44:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:44:40.578738 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51f1b6-192e-48e3-864f-4900f07e92f7 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:40.580390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ef51f1b6-192e-48e3-864f-4900f07e92f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:40.593459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51f1b6-192e-48e3-864f-4900f07e92f7 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:40.598295 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e0fe1-470d-467f-986f-cf62f49cb81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:40.599165 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e0fe1-470d-467f-986f-cf62f49cb81e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:40.599449 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e0fe1-470d-467f-986f-cf62f49cb81e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:40.599727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e0fe1-470d-467f-986f-cf62f49cb81e admin admin] PATH_INFO: `/v3/projects/de48895a84ba4614997d91e7881876ae` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:40.600953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e0fe1-470d-467f-986f-cf62f49cb81e admin admin] RBAC: Authorizing `identity:delete_project(project_id=de48895a84ba4614997d91e7881876ae)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:40.603002 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e0fe1-470d-467f-986f-cf62f49cb81e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:40.654720 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f12e0fe1-470d-467f-986f-cf62f49cb81e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de48895a84ba4614997d91e7881876ae', 'request_id': 'req-f12e0fe1-470d-467f-986f-cf62f49cb81e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:40.662365 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f12e0fe1-470d-467f-986f-cf62f49cb81e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de48895a84ba4614997d91e7881876ae', 'request_id': 'req-f12e0fe1-470d-467f-986f-cf62f49cb81e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:40.668993 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f12e0fe1-470d-467f-986f-cf62f49cb81e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de48895a84ba4614997d91e7881876ae', 'request_id': 'req-f12e0fe1-470d-467f-986f-cf62f49cb81e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:40.680023 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f12e0fe1-470d-467f-986f-cf62f49cb81e admin admin] The token cache is being invalidate because project de48895a84ba4614997d91e7881876ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:44:40.680306 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f12e0fe1-470d-467f-986f-cf62f49cb81e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:40.680931 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f12e0fe1-470d-467f-986f-cf62f49cb81e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:40.682825 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1254/2503] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:44:40 2022] DELETE /identity/v3/projects/de48895a84ba4614997d91e7881876ae => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:40.687824 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d667a4e-a271-426f-90c6-344bd4385a10 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:40.688196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d667a4e-a271-426f-90c6-344bd4385a10 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:40.688488 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d667a4e-a271-426f-90c6-344bd4385a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:40.706014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-1d667a4e-a271-426f-90c6-344bd4385a10 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:40.707378 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-1d667a4e-a271-426f-90c6-344bd4385a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:40.767662 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1250/2504] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:40.775381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb2c5d-601b-4323-baeb-d0c85dd61d6c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:40.775777 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb2c5d-601b-4323-baeb-d0c85dd61d6c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:40.776085 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb2c5d-601b-4323-baeb-d0c85dd61d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:40.795191 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-59bb2c5d-601b-4323-baeb-d0c85dd61d6c None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:40.796030 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-59bb2c5d-601b-4323-baeb-d0c85dd61d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:40.804127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1255/2505] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:40.808878 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d667a4e-a271-426f-90c6-344bd4385a10 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:40.818917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d667a4e-a271-426f-90c6-344bd4385a10 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:40.824174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ae301-f99c-4cd0-90d2-d728b70d4abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:40.824827 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ae301-f99c-4cd0-90d2-d728b70d4abf admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:40.825112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ae301-f99c-4cd0-90d2-d728b70d4abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:40.825506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ae301-f99c-4cd0-90d2-d728b70d4abf admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:40.826236 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ae301-f99c-4cd0-90d2-d728b70d4abf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:44:40.826540 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ae301-f99c-4cd0-90d2-d728b70d4abf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:40.828159 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ae301-f99c-4cd0-90d2-d728b70d4abf admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:40.839907 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1251/2506] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:44:40.844234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb2c5d-601b-4323-baeb-d0c85dd61d6c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:40.853501 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb2c5d-601b-4323-baeb-d0c85dd61d6c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:40.858419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfeb28-0269-4806-b1dc-52511af9553b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:40.859087 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfeb28-0269-4806-b1dc-52511af9553b admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:40.859352 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfeb28-0269-4806-b1dc-52511af9553b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:40.859625 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfeb28-0269-4806-b1dc-52511af9553b admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:40.860352 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cfeb28-0269-4806-b1dc-52511af9553b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:40.862247 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cfeb28-0269-4806-b1dc-52511af9553b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:40.891821 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1256/2507] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:44:40 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:44:40.895380 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ae301-f99c-4cd0-90d2-d728b70d4abf admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:40.905683 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ae301-f99c-4cd0-90d2-d728b70d4abf admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:40.963122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42622d78-3cd2-4a35-96f6-9f4bae859842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:40.964106 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42622d78-3cd2-4a35-96f6-9f4bae859842 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:40.964387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42622d78-3cd2-4a35-96f6-9f4bae859842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:40.964657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42622d78-3cd2-4a35-96f6-9f4bae859842 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:40.965359 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42622d78-3cd2-4a35-96f6-9f4bae859842 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:40.967245 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42622d78-3cd2-4a35-96f6-9f4bae859842 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:40.986934 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1252/2508] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:44:40 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:44:40.991042 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfeb28-0269-4806-b1dc-52511af9553b admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:41.001901 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfeb28-0269-4806-b1dc-52511af9553b admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:41.006611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bb316-3f93-424f-bbb8-b2a17adbeb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:41.007252 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bb316-3f93-424f-bbb8-b2a17adbeb5b admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:41.007531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bb316-3f93-424f-bbb8-b2a17adbeb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:41.007838 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bb316-3f93-424f-bbb8-b2a17adbeb5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:41.008549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bb316-3f93-424f-bbb8-b2a17adbeb5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:41.010194 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bb316-3f93-424f-bbb8-b2a17adbeb5b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:41.020429 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1257/2509] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:41.024196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42622d78-3cd2-4a35-96f6-9f4bae859842 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:41.034366 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42622d78-3cd2-4a35-96f6-9f4bae859842 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:41.038907 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc835937-0dee-4c0e-a329-211edde4677a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:41.039555 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc835937-0dee-4c0e-a329-211edde4677a admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:41.039872 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc835937-0dee-4c0e-a329-211edde4677a admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:41.040151 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc835937-0dee-4c0e-a329-211edde4677a admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:41.040860 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc835937-0dee-4c0e-a329-211edde4677a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:41.043465 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc835937-0dee-4c0e-a329-211edde4677a admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:41.053160 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1253/2510] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:41.057886 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bb316-3f93-424f-bbb8-b2a17adbeb5b admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:41.064103 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:41.068144 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bb316-3f93-424f-bbb8-b2a17adbeb5b admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:41.074394 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07558e-0826-46fe-be1f-3b50fad312c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:41.075414 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07558e-0826-46fe-be1f-3b50fad312c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:41.075943 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07558e-0826-46fe-be1f-3b50fad312c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:41.076383 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07558e-0826-46fe-be1f-3b50fad312c9 admin admin] PATH_INFO: `/v3/projects/3585575add99438c96529b88eaac45ad/users/d4ba8859abb94022beba9e9d6b0fdcb3/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:41.090716 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07558e-0826-46fe-be1f-3b50fad312c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3585575add99438c96529b88eaac45ad, user_id=d4ba8859abb94022beba9e9d6b0fdcb3, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:41.093319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07558e-0826-46fe-be1f-3b50fad312c9 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:41.105992 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1258/2511] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:44:41 2022] PUT /identity/v3/projects/3585575add99438c96529b88eaac45ad/users/d4ba8859abb94022beba9e9d6b0fdcb3/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:41.117299 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b4165-96b1-43cf-9ba6-997157e5c490 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:41.117591 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b4165-96b1-43cf-9ba6-997157e5c490 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:41.117873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b4165-96b1-43cf-9ba6-997157e5c490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:41.145111 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-1c8b4165-96b1-43cf-9ba6-997157e5c490 None None] MFA Rules not processed for user `d4ba8859abb94022beba9e9d6b0fdcb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:41.146327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-1c8b4165-96b1-43cf-9ba6-997157e5c490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:41.158260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:41.211792 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1254/2512] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:41.216349 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07558e-0826-46fe-be1f-3b50fad312c9 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:41.278518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07558e-0826-46fe-be1f-3b50fad312c9 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:41.283272 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc248da-affa-464a-a0e1-4cda54d898cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:41.283956 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc248da-affa-464a-a0e1-4cda54d898cf admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:41.284242 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc248da-affa-464a-a0e1-4cda54d898cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:41.284519 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc248da-affa-464a-a0e1-4cda54d898cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:41.285213 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc248da-affa-464a-a0e1-4cda54d898cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:41.287108 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc248da-affa-464a-a0e1-4cda54d898cf admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:41.318297 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1259/2513] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:44:41 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:44:41.320219 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b4165-96b1-43cf-9ba6-997157e5c490 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:41.331993 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b4165-96b1-43cf-9ba6-997157e5c490 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:41.388939 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce08334-5f2b-4f20-acae-d34c0d96bc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:41.389502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce08334-5f2b-4f20-acae-d34c0d96bc87 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:41.389790 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce08334-5f2b-4f20-acae-d34c0d96bc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:41.390063 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce08334-5f2b-4f20-acae-d34c0d96bc87 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:41.390774 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce08334-5f2b-4f20-acae-d34c0d96bc87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:41.392717 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce08334-5f2b-4f20-acae-d34c0d96bc87 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:41.412949 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1255/2514] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:44:41 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:44:41.417965 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc248da-affa-464a-a0e1-4cda54d898cf admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:41.428370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc248da-affa-464a-a0e1-4cda54d898cf admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:41.433678 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9726c7-e3ff-49ae-a75c-7201010e4e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:41.434432 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9726c7-e3ff-49ae-a75c-7201010e4e7d admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:41.434786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9726c7-e3ff-49ae-a75c-7201010e4e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:41.434973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9726c7-e3ff-49ae-a75c-7201010e4e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:41.435697 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9726c7-e3ff-49ae-a75c-7201010e4e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:41.437327 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9726c7-e3ff-49ae-a75c-7201010e4e7d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:41.448179 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1260/2515] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:41.452211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce08334-5f2b-4f20-acae-d34c0d96bc87 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:41.462419 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce08334-5f2b-4f20-acae-d34c0d96bc87 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:41.468227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d5f40-0595-4052-8635-a806a361c96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:41.469112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d5f40-0595-4052-8635-a806a361c96b admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:41.469435 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d5f40-0595-4052-8635-a806a361c96b admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:41.469713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d5f40-0595-4052-8635-a806a361c96b admin admin] PATH_INFO: `/v3/projects/9b14a5e2fc864328a8a8c9b5c7ce663f/users/5d4d82b820c0480cb2af2542f5c1b211/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:41.486130 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d5f40-0595-4052-8635-a806a361c96b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b14a5e2fc864328a8a8c9b5c7ce663f, user_id=5d4d82b820c0480cb2af2542f5c1b211, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:41.488818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d5f40-0595-4052-8635-a806a361c96b admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:41.502425 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1256/2516] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:44:41 2022] PUT /identity/v3/projects/9b14a5e2fc864328a8a8c9b5c7ce663f/users/5d4d82b820c0480cb2af2542f5c1b211/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:41.506388 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9726c7-e3ff-49ae-a75c-7201010e4e7d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:41.517141 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9726c7-e3ff-49ae-a75c-7201010e4e7d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:41.573490 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5894d2-2fcf-45ef-9fa2-74ca0bcf4395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:41.574158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5894d2-2fcf-45ef-9fa2-74ca0bcf4395 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:41.574446 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5894d2-2fcf-45ef-9fa2-74ca0bcf4395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:41.574724 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5894d2-2fcf-45ef-9fa2-74ca0bcf4395 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:41.575445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5894d2-2fcf-45ef-9fa2-74ca0bcf4395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:41.577087 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5894d2-2fcf-45ef-9fa2-74ca0bcf4395 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:41.587517 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1261/2517] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:41.592160 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d5f40-0595-4052-8635-a806a361c96b admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:41.602357 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d5f40-0595-4052-8635-a806a361c96b admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:41.607892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ae8c7-cd99-4c15-90f5-c0e2c30205a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:41.608751 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ae8c7-cd99-4c15-90f5-c0e2c30205a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:41.609035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ae8c7-cd99-4c15-90f5-c0e2c30205a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:41.609315 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ae8c7-cd99-4c15-90f5-c0e2c30205a7 admin admin] PATH_INFO: `/v3/projects/9b14a5e2fc864328a8a8c9b5c7ce663f/users/5d4d82b820c0480cb2af2542f5c1b211/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:41.611250 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ae8c7-cd99-4c15-90f5-c0e2c30205a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b14a5e2fc864328a8a8c9b5c7ce663f, user_id=5d4d82b820c0480cb2af2542f5c1b211, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:41.613911 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ae8c7-cd99-4c15-90f5-c0e2c30205a7 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:41.624180 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1257/2518] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:44:41 2022] PUT /identity/v3/projects/9b14a5e2fc864328a8a8c9b5c7ce663f/users/5d4d82b820c0480cb2af2542f5c1b211/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:41.634208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac75ba5-10d7-481f-b3e2-274f68f8f842 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:41.634502 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac75ba5-10d7-481f-b3e2-274f68f8f842 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:41.635279 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac75ba5-10d7-481f-b3e2-274f68f8f842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:41.661890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9ac75ba5-10d7-481f-b3e2-274f68f8f842 None None] MFA Rules not processed for user `5d4d82b820c0480cb2af2542f5c1b211`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:41.663024 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9ac75ba5-10d7-481f-b3e2-274f68f8f842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:41.702281 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:41.732404 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1262/2519] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:41 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:41.743009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ae8c7-cd99-4c15-90f5-c0e2c30205a7 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:41.744811 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-d71ae8c7-cd99-4c15-90f5-c0e2c30205a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:41.801766 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ae8c7-cd99-4c15-90f5-c0e2c30205a7 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:41.805424 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fea5a6-fec7-4045-a94f-fdfb7178c37c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:41.806053 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fea5a6-fec7-4045-a94f-fdfb7178c37c service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:41.806339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fea5a6-fec7-4045-a94f-fdfb7178c37c service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:41.806633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fea5a6-fec7-4045-a94f-fdfb7178c37c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:41.826498 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:41.864858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fea5a6-fec7-4045-a94f-fdfb7178c37c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:41.866291 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fea5a6-fec7-4045-a94f-fdfb7178c37c service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:41.875243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1258/2520] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:44:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:44:44.403986 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac75ba5-10d7-481f-b3e2-274f68f8f842 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:44.470830 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac75ba5-10d7-481f-b3e2-274f68f8f842 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:44.475248 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b0cff7-4701-4c33-b616-2962315470a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:44.476158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0cff7-4701-4c33-b616-2962315470a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:44.476438 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0cff7-4701-4c33-b616-2962315470a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:44.476705 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0cff7-4701-4c33-b616-2962315470a4 admin admin] PATH_INFO: `/v3/users/d4ba8859abb94022beba9e9d6b0fdcb3` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:44.477880 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b0cff7-4701-4c33-b616-2962315470a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4ba8859abb94022beba9e9d6b0fdcb3)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:44.480136 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b0cff7-4701-4c33-b616-2962315470a4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:44.527276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-41b0cff7-4701-4c33-b616-2962315470a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4ba8859abb94022beba9e9d6b0fdcb3'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:44.534993 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-41b0cff7-4701-4c33-b616-2962315470a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4ba8859abb94022beba9e9d6b0fdcb3'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:44.535688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-41b0cff7-4701-4c33-b616-2962315470a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4ba8859abb94022beba9e9d6b0fdcb3'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:44.536410 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-41b0cff7-4701-4c33-b616-2962315470a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4ba8859abb94022beba9e9d6b0fdcb3'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:44.548513 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-41b0cff7-4701-4c33-b616-2962315470a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4ba8859abb94022beba9e9d6b0fdcb3'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:44.562197 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1263/2521] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:44:44 2022] DELETE /identity/v3/users/d4ba8859abb94022beba9e9d6b0fdcb3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:44.566932 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89103f-1883-4fc2-83aa-085d469c3a4f None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:44.567390 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89103f-1883-4fc2-83aa-085d469c3a4f None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:44.567496 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89103f-1883-4fc2-83aa-085d469c3a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:44.585713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-cd89103f-1883-4fc2-83aa-085d469c3a4f None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:44.586955 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-cd89103f-1883-4fc2-83aa-085d469c3a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:44.650391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1259/2522] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:44:44.660701 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b0cff7-4701-4c33-b616-2962315470a4 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:44.662360 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-41b0cff7-4701-4c33-b616-2962315470a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:44.721588 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b0cff7-4701-4c33-b616-2962315470a4 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:44.725199 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f93d7b-515a-462e-bb24-ab3a83ffb84e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:44.725846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f93d7b-515a-462e-bb24-ab3a83ffb84e service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:44.726131 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f93d7b-515a-462e-bb24-ab3a83ffb84e service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:44.726408 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f93d7b-515a-462e-bb24-ab3a83ffb84e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:44.737123 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f93d7b-515a-462e-bb24-ab3a83ffb84e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:44.738598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f93d7b-515a-462e-bb24-ab3a83ffb84e service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:44.742620 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:44.752117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1264/2523] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:44:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:44:45.033577 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89103f-1883-4fc2-83aa-085d469c3a4f None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:45.035177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-cd89103f-1883-4fc2-83aa-085d469c3a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:45.047809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89103f-1883-4fc2-83aa-085d469c3a4f None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:45.053127 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f180ae-57cb-445c-b425-211b8870509b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:45.054021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f180ae-57cb-445c-b425-211b8870509b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:45.054338 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f180ae-57cb-445c-b425-211b8870509b admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:45.054621 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f180ae-57cb-445c-b425-211b8870509b admin admin] PATH_INFO: `/v3/projects/3585575add99438c96529b88eaac45ad` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:45.055858 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f180ae-57cb-445c-b425-211b8870509b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3585575add99438c96529b88eaac45ad)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:45.058584 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f180ae-57cb-445c-b425-211b8870509b admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:45.110947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-12f180ae-57cb-445c-b425-211b8870509b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3585575add99438c96529b88eaac45ad', 'request_id': 'req-12f180ae-57cb-445c-b425-211b8870509b'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:45.116642 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-12f180ae-57cb-445c-b425-211b8870509b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3585575add99438c96529b88eaac45ad', 'request_id': 'req-12f180ae-57cb-445c-b425-211b8870509b'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:45.125048 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-12f180ae-57cb-445c-b425-211b8870509b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3585575add99438c96529b88eaac45ad', 'request_id': 'req-12f180ae-57cb-445c-b425-211b8870509b'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:45.138075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-12f180ae-57cb-445c-b425-211b8870509b admin admin] The token cache is being invalidate because project 3585575add99438c96529b88eaac45ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:44:45.138376 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-12f180ae-57cb-445c-b425-211b8870509b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:45.139322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-12f180ae-57cb-445c-b425-211b8870509b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:45.141206 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1260/2524] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:44:45 2022] DELETE /identity/v3/projects/3585575add99438c96529b88eaac45ad => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:45.145116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f93d7b-515a-462e-bb24-ab3a83ffb84e service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:45.155824 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f93d7b-515a-462e-bb24-ab3a83ffb84e service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:45.216248 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8076a7b-95f9-446e-8f94-a924ebf1481e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:45.217173 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8076a7b-95f9-446e-8f94-a924ebf1481e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:45.217454 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8076a7b-95f9-446e-8f94-a924ebf1481e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:45.217725 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8076a7b-95f9-446e-8f94-a924ebf1481e admin admin] PATH_INFO: `/v3/users/5d4d82b820c0480cb2af2542f5c1b211` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:45.218905 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8076a7b-95f9-446e-8f94-a924ebf1481e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d4d82b820c0480cb2af2542f5c1b211)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:45.220878 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8076a7b-95f9-446e-8f94-a924ebf1481e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:45.271437 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d8076a7b-95f9-446e-8f94-a924ebf1481e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d4d82b820c0480cb2af2542f5c1b211'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:45.279672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d8076a7b-95f9-446e-8f94-a924ebf1481e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d4d82b820c0480cb2af2542f5c1b211'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:45.280702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d8076a7b-95f9-446e-8f94-a924ebf1481e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d4d82b820c0480cb2af2542f5c1b211'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:45.281598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d8076a7b-95f9-446e-8f94-a924ebf1481e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d4d82b820c0480cb2af2542f5c1b211'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:45.295570 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d8076a7b-95f9-446e-8f94-a924ebf1481e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d4d82b820c0480cb2af2542f5c1b211'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:45.308771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1265/2525] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:44:45 2022] DELETE /identity/v3/users/5d4d82b820c0480cb2af2542f5c1b211 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:45.623597 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f180ae-57cb-445c-b425-211b8870509b admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:45.625276 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-12f180ae-57cb-445c-b425-211b8870509b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:45.641524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f180ae-57cb-445c-b425-211b8870509b admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:45.694771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:45.712844 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b0014-9ab1-4a66-91d0-bd0c3d1eda0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:45.713460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b0014-9ab1-4a66-91d0-bd0c3d1eda0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:45.713848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b0014-9ab1-4a66-91d0-bd0c3d1eda0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:45.714229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b0014-9ab1-4a66-91d0-bd0c3d1eda0d admin admin] PATH_INFO: `/v3/projects/9b14a5e2fc864328a8a8c9b5c7ce663f` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:45.715802 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b0014-9ab1-4a66-91d0-bd0c3d1eda0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b14a5e2fc864328a8a8c9b5c7ce663f)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:45.717818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b0014-9ab1-4a66-91d0-bd0c3d1eda0d admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:45.773268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-452b0014-9ab1-4a66-91d0-bd0c3d1eda0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b14a5e2fc864328a8a8c9b5c7ce663f', 'request_id': 'req-452b0014-9ab1-4a66-91d0-bd0c3d1eda0d'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:45.780808 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-452b0014-9ab1-4a66-91d0-bd0c3d1eda0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b14a5e2fc864328a8a8c9b5c7ce663f', 'request_id': 'req-452b0014-9ab1-4a66-91d0-bd0c3d1eda0d'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:45.788867 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-452b0014-9ab1-4a66-91d0-bd0c3d1eda0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b14a5e2fc864328a8a8c9b5c7ce663f', 'request_id': 'req-452b0014-9ab1-4a66-91d0-bd0c3d1eda0d'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:45.800869 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-452b0014-9ab1-4a66-91d0-bd0c3d1eda0d admin admin] The token cache is being invalidate because project 9b14a5e2fc864328a8a8c9b5c7ce663f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:44:45.801408 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-452b0014-9ab1-4a66-91d0-bd0c3d1eda0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:45.802149 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-452b0014-9ab1-4a66-91d0-bd0c3d1eda0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:45.804067 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1261/2526] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:44:45 2022] DELETE /identity/v3/projects/9b14a5e2fc864328a8a8c9b5c7ce663f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:45.809233 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ed4d4-ce77-4351-91e2-8bcc75af0216 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:45.809759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ed4d4-ce77-4351-91e2-8bcc75af0216 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:45.809857 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ed4d4-ce77-4351-91e2-8bcc75af0216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:45.829137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-972ed4d4-ce77-4351-91e2-8bcc75af0216 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:45.830495 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-972ed4d4-ce77-4351-91e2-8bcc75af0216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:45.905140 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1266/2527] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:45.915479 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fba15f-ba77-4e69-bb65-8cec58ef03e2 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:45.917087 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fba15f-ba77-4e69-bb65-8cec58ef03e2 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:45.917638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fba15f-ba77-4e69-bb65-8cec58ef03e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:45.946749 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-e4fba15f-ba77-4e69-bb65-8cec58ef03e2 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:45.948670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-e4fba15f-ba77-4e69-bb65-8cec58ef03e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:45.957962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1262/2528] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:45.962214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ed4d4-ce77-4351-91e2-8bcc75af0216 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:45.973303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ed4d4-ce77-4351-91e2-8bcc75af0216 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:45.978583 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96664bd6-a63b-452f-984a-ccd801c66491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:45.979342 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96664bd6-a63b-452f-984a-ccd801c66491 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:45.979752 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96664bd6-a63b-452f-984a-ccd801c66491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:45.980065 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96664bd6-a63b-452f-984a-ccd801c66491 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:45.981296 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96664bd6-a63b-452f-984a-ccd801c66491 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:44:45.981296 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96664bd6-a63b-452f-984a-ccd801c66491 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:45.982822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96664bd6-a63b-452f-984a-ccd801c66491 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:45.995820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1267/2529] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:44:46.000766 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fba15f-ba77-4e69-bb65-8cec58ef03e2 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:46.010256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fba15f-ba77-4e69-bb65-8cec58ef03e2 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:46.015643 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd4d123-1b83-4606-8f1e-8d26315ac7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:46.016350 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4d123-1b83-4606-8f1e-8d26315ac7dd admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:46.016629 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4d123-1b83-4606-8f1e-8d26315ac7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:46.016915 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4d123-1b83-4606-8f1e-8d26315ac7dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:46.017656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd4d123-1b83-4606-8f1e-8d26315ac7dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:46.019846 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd4d123-1b83-4606-8f1e-8d26315ac7dd admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:46.048626 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1263/2530] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:44:45 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:44:46.052799 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96664bd6-a63b-452f-984a-ccd801c66491 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:46.060025 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:46.064372 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96664bd6-a63b-452f-984a-ccd801c66491 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:46.123495 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9e083-f490-4ab4-b578-a3da3c1aa08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:46.124202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9e083-f490-4ab4-b578-a3da3c1aa08d admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:46.124489 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9e083-f490-4ab4-b578-a3da3c1aa08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:46.124759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9e083-f490-4ab4-b578-a3da3c1aa08d admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:46.125457 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9e083-f490-4ab4-b578-a3da3c1aa08d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:46.127367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9e083-f490-4ab4-b578-a3da3c1aa08d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:46.148603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1268/2531] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:44:46 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:44:46.152706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd4d123-1b83-4606-8f1e-8d26315ac7dd admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:46.163584 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd4d123-1b83-4606-8f1e-8d26315ac7dd admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:46.169519 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa33187-6e17-443a-b16f-0a4495f5c3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:46.170173 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa33187-6e17-443a-b16f-0a4495f5c3e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:46.170446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa33187-6e17-443a-b16f-0a4495f5c3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:46.170724 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa33187-6e17-443a-b16f-0a4495f5c3e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:46.171453 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa33187-6e17-443a-b16f-0a4495f5c3e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:46.173082 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa33187-6e17-443a-b16f-0a4495f5c3e4 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:46.184832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1264/2532] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:46.189090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9e083-f490-4ab4-b578-a3da3c1aa08d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:46.199824 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9e083-f490-4ab4-b578-a3da3c1aa08d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:46.205063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cbd4b7-4515-45d9-9afa-8d3cbaa231bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:46.205752 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbd4b7-4515-45d9-9afa-8d3cbaa231bb admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:46.206038 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbd4b7-4515-45d9-9afa-8d3cbaa231bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:46.206317 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbd4b7-4515-45d9-9afa-8d3cbaa231bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:46.207057 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cbd4b7-4515-45d9-9afa-8d3cbaa231bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:46.208724 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cbd4b7-4515-45d9-9afa-8d3cbaa231bb admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:46.222648 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1269/2533] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:46.226776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa33187-6e17-443a-b16f-0a4495f5c3e4 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:46.237930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa33187-6e17-443a-b16f-0a4495f5c3e4 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:46.242656 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faedbf1-03e7-47f5-aeea-76fb38f480d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:46.243653 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faedbf1-03e7-47f5-aeea-76fb38f480d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:46.243818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faedbf1-03e7-47f5-aeea-76fb38f480d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:46.244088 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faedbf1-03e7-47f5-aeea-76fb38f480d3 admin admin] PATH_INFO: `/v3/projects/0289f81a823b41c7abdd9efe407a2300/users/305f3dcde48f4d8e989f3f91ee2ada74/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:46.260479 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faedbf1-03e7-47f5-aeea-76fb38f480d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0289f81a823b41c7abdd9efe407a2300, user_id=305f3dcde48f4d8e989f3f91ee2ada74, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:46.263606 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faedbf1-03e7-47f5-aeea-76fb38f480d3 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:46.277940 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1265/2534] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:44:46 2022] PUT /identity/v3/projects/0289f81a823b41c7abdd9efe407a2300/users/305f3dcde48f4d8e989f3f91ee2ada74/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:46.289521 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47ac7f-96e5-41d8-8d1d-968e9e6e471b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:46.289840 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47ac7f-96e5-41d8-8d1d-968e9e6e471b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:46.290129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47ac7f-96e5-41d8-8d1d-968e9e6e471b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:46.319317 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-da47ac7f-96e5-41d8-8d1d-968e9e6e471b None None] MFA Rules not processed for user `305f3dcde48f4d8e989f3f91ee2ada74`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:46.320775 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-da47ac7f-96e5-41d8-8d1d-968e9e6e471b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:46.404304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1270/2535] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:44:46.408800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faedbf1-03e7-47f5-aeea-76fb38f480d3 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:46.474104 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faedbf1-03e7-47f5-aeea-76fb38f480d3 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:46.478887 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c311e03-323d-413e-9793-a5f7641ca594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:46.479539 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c311e03-323d-413e-9793-a5f7641ca594 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:46.479876 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c311e03-323d-413e-9793-a5f7641ca594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:46.480151 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c311e03-323d-413e-9793-a5f7641ca594 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:46.480936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c311e03-323d-413e-9793-a5f7641ca594 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:46.483244 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c311e03-323d-413e-9793-a5f7641ca594 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:46.497484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:46.514526 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1266/2536] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:44:46 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:44:46.519735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47ac7f-96e5-41d8-8d1d-968e9e6e471b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:46.531879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47ac7f-96e5-41d8-8d1d-968e9e6e471b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:46.590112 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6eb3e-6742-4f3d-966b-f2e6198e1b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:46.590695 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6eb3e-6742-4f3d-966b-f2e6198e1b96 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:46.590978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6eb3e-6742-4f3d-966b-f2e6198e1b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:46.591288 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6eb3e-6742-4f3d-966b-f2e6198e1b96 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:46.591995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6eb3e-6742-4f3d-966b-f2e6198e1b96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:46.593991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6eb3e-6742-4f3d-966b-f2e6198e1b96 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:46.613584 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1271/2537] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:44:46 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:44:46.617574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c311e03-323d-413e-9793-a5f7641ca594 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:46.628637 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c311e03-323d-413e-9793-a5f7641ca594 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:46.633245 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036dc45-b2ce-42f8-a2ae-d8853a0a5c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:46.633903 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036dc45-b2ce-42f8-a2ae-d8853a0a5c80 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:46.634189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036dc45-b2ce-42f8-a2ae-d8853a0a5c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:46.634471 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036dc45-b2ce-42f8-a2ae-d8853a0a5c80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:46.635881 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036dc45-b2ce-42f8-a2ae-d8853a0a5c80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:46.637479 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036dc45-b2ce-42f8-a2ae-d8853a0a5c80 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:46.648301 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1267/2538] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:46.652290 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6eb3e-6742-4f3d-966b-f2e6198e1b96 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:46.657426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:46.663648 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6eb3e-6742-4f3d-966b-f2e6198e1b96 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:46.668815 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d835576a-5485-4a65-a27f-199bd529cc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:46.669482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d835576a-5485-4a65-a27f-199bd529cc14 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:46.669768 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d835576a-5485-4a65-a27f-199bd529cc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:46.670045 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d835576a-5485-4a65-a27f-199bd529cc14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:46.670771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d835576a-5485-4a65-a27f-199bd529cc14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:46.672435 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d835576a-5485-4a65-a27f-199bd529cc14 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:46.683029 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1272/2539] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:46.687318 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036dc45-b2ce-42f8-a2ae-d8853a0a5c80 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:46.697782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036dc45-b2ce-42f8-a2ae-d8853a0a5c80 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:46.702614 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4a7b8-c013-4a38-810e-bcb21151a33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:46.703546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4a7b8-c013-4a38-810e-bcb21151a33c admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:46.703857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4a7b8-c013-4a38-810e-bcb21151a33c admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:46.704157 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4a7b8-c013-4a38-810e-bcb21151a33c admin admin] PATH_INFO: `/v3/projects/34a7cbd58e5949eb85af78c5c65c606d/users/8cea53b03eaa4471a23065ff34a4dfc4/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:46.718562 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4a7b8-c013-4a38-810e-bcb21151a33c admin admin] RBAC: Authorizing `identity:create_grant(project_id=34a7cbd58e5949eb85af78c5c65c606d, user_id=8cea53b03eaa4471a23065ff34a4dfc4, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:46.721252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4a7b8-c013-4a38-810e-bcb21151a33c admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:46.734054 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1268/2540] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:44:46 2022] PUT /identity/v3/projects/34a7cbd58e5949eb85af78c5c65c606d/users/8cea53b03eaa4471a23065ff34a4dfc4/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:44:46.744955 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1c566-abf0-4893-b370-8ca73d7530ff None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:46.744955 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1c566-abf0-4893-b370-8ca73d7530ff None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:46.745266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1c566-abf0-4893-b370-8ca73d7530ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:46.771561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3ff1c566-abf0-4893-b370-8ca73d7530ff None None] MFA Rules not processed for user `8cea53b03eaa4471a23065ff34a4dfc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:46.772796 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3ff1c566-abf0-4893-b370-8ca73d7530ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:46.835390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1273/2541] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:46.839872 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4a7b8-c013-4a38-810e-bcb21151a33c admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:46.902078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4a7b8-c013-4a38-810e-bcb21151a33c admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:46.906713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588afb51-4249-4f80-b450-4c3764815a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:46.907356 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588afb51-4249-4f80-b450-4c3764815a9d admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:46.907642 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588afb51-4249-4f80-b450-4c3764815a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:46.907956 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588afb51-4249-4f80-b450-4c3764815a9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:46.908659 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588afb51-4249-4f80-b450-4c3764815a9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:46.910542 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588afb51-4249-4f80-b450-4c3764815a9d admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:46.936156 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1269/2542] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:44:46 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:44:46.940227 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1c566-abf0-4893-b370-8ca73d7530ff None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:46.950873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1c566-abf0-4893-b370-8ca73d7530ff None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:47.006740 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0d8c9-bc82-43d7-87af-b22a82893cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:47.007402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0d8c9-bc82-43d7-87af-b22a82893cf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:47.007685 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0d8c9-bc82-43d7-87af-b22a82893cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:47.007988 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0d8c9-bc82-43d7-87af-b22a82893cf3 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:47.008677 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0d8c9-bc82-43d7-87af-b22a82893cf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:47.010719 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0d8c9-bc82-43d7-87af-b22a82893cf3 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:47.031138 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1274/2543] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:44:46 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:44:47.035102 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588afb51-4249-4f80-b450-4c3764815a9d admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:47.045865 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588afb51-4249-4f80-b450-4c3764815a9d admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:47.050530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f773d-5615-46e4-80e7-f1d44c70e0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:47.051179 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f773d-5615-46e4-80e7-f1d44c70e0b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:47.051464 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f773d-5615-46e4-80e7-f1d44c70e0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:47.051741 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f773d-5615-46e4-80e7-f1d44c70e0b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:47.052486 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f773d-5615-46e4-80e7-f1d44c70e0b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:47.054070 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f773d-5615-46e4-80e7-f1d44c70e0b7 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:47.065111 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1270/2544] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:47.069219 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0d8c9-bc82-43d7-87af-b22a82893cf3 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:47.079041 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0d8c9-bc82-43d7-87af-b22a82893cf3 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:47.083775 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db69dc4-accd-43e0-8964-1c7abf94fd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:47.084826 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db69dc4-accd-43e0-8964-1c7abf94fd6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:47.085148 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db69dc4-accd-43e0-8964-1c7abf94fd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:47.085493 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db69dc4-accd-43e0-8964-1c7abf94fd6f admin admin] PATH_INFO: `/v3/projects/8ba286a5d80b4a048ee93e18202722f0/users/ee42a33caed34a14964801a91a06ece2/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:47.100062 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db69dc4-accd-43e0-8964-1c7abf94fd6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba286a5d80b4a048ee93e18202722f0, user_id=ee42a33caed34a14964801a91a06ece2, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:47.102654 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db69dc4-accd-43e0-8964-1c7abf94fd6f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:47.115526 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1275/2545] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:44:47 2022] PUT /identity/v3/projects/8ba286a5d80b4a048ee93e18202722f0/users/ee42a33caed34a14964801a91a06ece2/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:47.120045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f773d-5615-46e4-80e7-f1d44c70e0b7 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:47.129729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f773d-5615-46e4-80e7-f1d44c70e0b7 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:47.185930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6066dabb-2d9a-4a5f-b39a-32c8d293f66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:47.186657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066dabb-2d9a-4a5f-b39a-32c8d293f66b admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:47.186943 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066dabb-2d9a-4a5f-b39a-32c8d293f66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:47.187220 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066dabb-2d9a-4a5f-b39a-32c8d293f66b admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:47.187969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6066dabb-2d9a-4a5f-b39a-32c8d293f66b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:47.189550 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6066dabb-2d9a-4a5f-b39a-32c8d293f66b admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:47.199368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1271/2546] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:47.203501 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db69dc4-accd-43e0-8964-1c7abf94fd6f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:47.213555 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db69dc4-accd-43e0-8964-1c7abf94fd6f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:47.218138 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a25ea3-1f02-47c0-a520-bfcd99cf9822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:47.219001 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a25ea3-1f02-47c0-a520-bfcd99cf9822 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:47.219280 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a25ea3-1f02-47c0-a520-bfcd99cf9822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:47.219554 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a25ea3-1f02-47c0-a520-bfcd99cf9822 admin admin] PATH_INFO: `/v3/projects/8ba286a5d80b4a048ee93e18202722f0/users/ee42a33caed34a14964801a91a06ece2/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:47.221628 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a25ea3-1f02-47c0-a520-bfcd99cf9822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba286a5d80b4a048ee93e18202722f0, user_id=ee42a33caed34a14964801a91a06ece2, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:47.224277 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a25ea3-1f02-47c0-a520-bfcd99cf9822 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:47.234918 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1276/2547] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:44:47 2022] PUT /identity/v3/projects/8ba286a5d80b4a048ee93e18202722f0/users/ee42a33caed34a14964801a91a06ece2/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:44:47.246703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1101dc-52a3-4d76-9acf-c4870df1a1f4 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:47.246996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1101dc-52a3-4d76-9acf-c4870df1a1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:47.247303 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1101dc-52a3-4d76-9acf-c4870df1a1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:47.274292 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-cc1101dc-52a3-4d76-9acf-c4870df1a1f4 None None] MFA Rules not processed for user `ee42a33caed34a14964801a91a06ece2`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:47.276194 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-cc1101dc-52a3-4d76-9acf-c4870df1a1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:47.295303 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:47.346570 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1272/2548] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:47.358455 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a25ea3-1f02-47c0-a520-bfcd99cf9822 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:47.360199 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c3a25ea3-1f02-47c0-a520-bfcd99cf9822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:47.396676 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:47.420366 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a25ea3-1f02-47c0-a520-bfcd99cf9822 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:47.423953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1ee24-f01e-44bd-bf8f-3fbf43238cba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:47.424586 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1ee24-f01e-44bd-bf8f-3fbf43238cba service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:47.424862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1ee24-f01e-44bd-bf8f-3fbf43238cba service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:47.425136 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1ee24-f01e-44bd-bf8f-3fbf43238cba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:47.481719 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1ee24-f01e-44bd-bf8f-3fbf43238cba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:47.483073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1ee24-f01e-44bd-bf8f-3fbf43238cba service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:47.491924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1277/2549] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:44:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:44:49.016047 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1101dc-52a3-4d76-9acf-c4870df1a1f4 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:49.017457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-cc1101dc-52a3-4d76-9acf-c4870df1a1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:49.029724 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1101dc-52a3-4d76-9acf-c4870df1a1f4 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:49.033281 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d3ed0-92b1-44e6-8205-ba67de70c385 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:49.033912 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d3ed0-92b1-44e6-8205-ba67de70c385 service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:49.034197 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d3ed0-92b1-44e6-8205-ba67de70c385 service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:49.034473 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d3ed0-92b1-44e6-8205-ba67de70c385 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:49.090511 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d3ed0-92b1-44e6-8205-ba67de70c385 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:49.091888 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d3ed0-92b1-44e6-8205-ba67de70c385 service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:49.100729 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1273/2550] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:44:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:44:49.598619 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1ee24-f01e-44bd-bf8f-3fbf43238cba service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:49.607629 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1ee24-f01e-44bd-bf8f-3fbf43238cba service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:49.611479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79556293-a546-4636-bf05-44b2019ecb8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:49.612141 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79556293-a546-4636-bf05-44b2019ecb8c service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:49.612449 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79556293-a546-4636-bf05-44b2019ecb8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:49.612721 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79556293-a546-4636-bf05-44b2019ecb8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:49.621957 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79556293-a546-4636-bf05-44b2019ecb8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:49.623364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79556293-a546-4636-bf05-44b2019ecb8c service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:49.632630 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1278/2551] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:44:49 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:44:53.443419 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d3ed0-92b1-44e6-8205-ba67de70c385 service cinder] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:53.508083 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d3ed0-92b1-44e6-8205-ba67de70c385 service cinder] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:53.514136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80898a-fcbb-4e4b-84d2-15bffb4f9b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:53.515035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80898a-fcbb-4e4b-84d2-15bffb4f9b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:53.515322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80898a-fcbb-4e4b-84d2-15bffb4f9b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:53.515595 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80898a-fcbb-4e4b-84d2-15bffb4f9b4d admin admin] PATH_INFO: `/v3/users/305f3dcde48f4d8e989f3f91ee2ada74` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:53.516808 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80898a-fcbb-4e4b-84d2-15bffb4f9b4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=305f3dcde48f4d8e989f3f91ee2ada74)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:53.518707 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80898a-fcbb-4e4b-84d2-15bffb4f9b4d admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:53.578558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-ee80898a-fcbb-4e4b-84d2-15bffb4f9b4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '305f3dcde48f4d8e989f3f91ee2ada74'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:53.585864 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-ee80898a-fcbb-4e4b-84d2-15bffb4f9b4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '305f3dcde48f4d8e989f3f91ee2ada74'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:53.587758 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:53.594678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-ee80898a-fcbb-4e4b-84d2-15bffb4f9b4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '305f3dcde48f4d8e989f3f91ee2ada74'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:53.595470 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-ee80898a-fcbb-4e4b-84d2-15bffb4f9b4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '305f3dcde48f4d8e989f3f91ee2ada74'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:53.609130 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-ee80898a-fcbb-4e4b-84d2-15bffb4f9b4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '305f3dcde48f4d8e989f3f91ee2ada74'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:53.615588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1274/2552] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:44:53 2022] DELETE /identity/v3/users/305f3dcde48f4d8e989f3f91ee2ada74 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:53.620919 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81b364-fe86-4d1f-8e80-5a72e3d95557 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:53.621395 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81b364-fe86-4d1f-8e80-5a72e3d95557 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:53.621490 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81b364-fe86-4d1f-8e80-5a72e3d95557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:53.639386 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ec81b364-fe86-4d1f-8e80-5a72e3d95557 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:53.640552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ec81b364-fe86-4d1f-8e80-5a72e3d95557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:53.701196 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1279/2553] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:53.710999 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80898a-fcbb-4e4b-84d2-15bffb4f9b4d admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:53.712510 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ee80898a-fcbb-4e4b-84d2-15bffb4f9b4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:53.770400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80898a-fcbb-4e4b-84d2-15bffb4f9b4d admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:53.774077 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e741d70-3898-40fb-abae-654634484a5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:53.774739 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e741d70-3898-40fb-abae-654634484a5a service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:53.775064 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e741d70-3898-40fb-abae-654634484a5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:53.775317 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e741d70-3898-40fb-abae-654634484a5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:53.784413 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e741d70-3898-40fb-abae-654634484a5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:53.785963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e741d70-3898-40fb-abae-654634484a5a service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:53.796367 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1275/2554] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:44:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:44:54.069319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81b364-fe86-4d1f-8e80-5a72e3d95557 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:54.070931 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ec81b364-fe86-4d1f-8e80-5a72e3d95557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:54.086139 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81b364-fe86-4d1f-8e80-5a72e3d95557 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:54.093136 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fec98-e839-4b2d-bd86-c911b6e49852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:54.093915 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fec98-e839-4b2d-bd86-c911b6e49852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:54.094200 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fec98-e839-4b2d-bd86-c911b6e49852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:54.094473 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fec98-e839-4b2d-bd86-c911b6e49852 admin admin] PATH_INFO: `/v3/projects/0289f81a823b41c7abdd9efe407a2300` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:54.095637 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fec98-e839-4b2d-bd86-c911b6e49852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0289f81a823b41c7abdd9efe407a2300)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:54.097548 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fec98-e839-4b2d-bd86-c911b6e49852 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:54.144466 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a13fec98-e839-4b2d-bd86-c911b6e49852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0289f81a823b41c7abdd9efe407a2300', 'request_id': 'req-a13fec98-e839-4b2d-bd86-c911b6e49852'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:54.152102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a13fec98-e839-4b2d-bd86-c911b6e49852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0289f81a823b41c7abdd9efe407a2300', 'request_id': 'req-a13fec98-e839-4b2d-bd86-c911b6e49852'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:54.157751 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a13fec98-e839-4b2d-bd86-c911b6e49852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0289f81a823b41c7abdd9efe407a2300', 'request_id': 'req-a13fec98-e839-4b2d-bd86-c911b6e49852'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:54.164822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:54.170874 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a13fec98-e839-4b2d-bd86-c911b6e49852 admin admin] The token cache is being invalidate because project 0289f81a823b41c7abdd9efe407a2300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:44:54.171327 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a13fec98-e839-4b2d-bd86-c911b6e49852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:54.172476 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a13fec98-e839-4b2d-bd86-c911b6e49852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:54.174241 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1280/2555] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:44:54 2022] DELETE /identity/v3/projects/0289f81a823b41c7abdd9efe407a2300 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:54.178167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e741d70-3898-40fb-abae-654634484a5a service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:54.188049 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e741d70-3898-40fb-abae-654634484a5a service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:54.245557 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb4d57-0309-4a1b-8a1c-b51c63731891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:54.246425 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb4d57-0309-4a1b-8a1c-b51c63731891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:54.246707 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb4d57-0309-4a1b-8a1c-b51c63731891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:54.246983 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb4d57-0309-4a1b-8a1c-b51c63731891 admin admin] PATH_INFO: `/v3/users/8cea53b03eaa4471a23065ff34a4dfc4` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:54.248203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb4d57-0309-4a1b-8a1c-b51c63731891 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cea53b03eaa4471a23065ff34a4dfc4)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:54.250107 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb4d57-0309-4a1b-8a1c-b51c63731891 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:54.295400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b3eb4d57-0309-4a1b-8a1c-b51c63731891 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cea53b03eaa4471a23065ff34a4dfc4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:54.302367 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b3eb4d57-0309-4a1b-8a1c-b51c63731891 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cea53b03eaa4471a23065ff34a4dfc4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:54.309801 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b3eb4d57-0309-4a1b-8a1c-b51c63731891 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cea53b03eaa4471a23065ff34a4dfc4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:54.310426 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b3eb4d57-0309-4a1b-8a1c-b51c63731891 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cea53b03eaa4471a23065ff34a4dfc4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:54.321558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:54.322928 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b3eb4d57-0309-4a1b-8a1c-b51c63731891 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cea53b03eaa4471a23065ff34a4dfc4'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:54.331106 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1276/2556] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:44:54 2022] DELETE /identity/v3/users/8cea53b03eaa4471a23065ff34a4dfc4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:54.605569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fec98-e839-4b2d-bd86-c911b6e49852 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:54.607090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a13fec98-e839-4b2d-bd86-c911b6e49852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:54.621451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fec98-e839-4b2d-bd86-c911b6e49852 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:54.679983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352db591-506a-4374-a38a-174a9f63ab2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:54.680782 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352db591-506a-4374-a38a-174a9f63ab2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:54.681058 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352db591-506a-4374-a38a-174a9f63ab2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:54.681505 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352db591-506a-4374-a38a-174a9f63ab2c admin admin] PATH_INFO: `/v3/projects/34a7cbd58e5949eb85af78c5c65c606d` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:54.682874 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352db591-506a-4374-a38a-174a9f63ab2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=34a7cbd58e5949eb85af78c5c65c606d)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:54.684803 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352db591-506a-4374-a38a-174a9f63ab2c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:54.730427 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-352db591-506a-4374-a38a-174a9f63ab2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34a7cbd58e5949eb85af78c5c65c606d', 'request_id': 'req-352db591-506a-4374-a38a-174a9f63ab2c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:54.738240 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-352db591-506a-4374-a38a-174a9f63ab2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34a7cbd58e5949eb85af78c5c65c606d', 'request_id': 'req-352db591-506a-4374-a38a-174a9f63ab2c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:54.743897 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-352db591-506a-4374-a38a-174a9f63ab2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34a7cbd58e5949eb85af78c5c65c606d', 'request_id': 'req-352db591-506a-4374-a38a-174a9f63ab2c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:54.754767 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-352db591-506a-4374-a38a-174a9f63ab2c admin admin] The token cache is being invalidate because project 34a7cbd58e5949eb85af78c5c65c606d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:44:54.755195 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-352db591-506a-4374-a38a-174a9f63ab2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:54.755881 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-352db591-506a-4374-a38a-174a9f63ab2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:54.757429 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1281/2557] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:44:54 2022] DELETE /identity/v3/projects/34a7cbd58e5949eb85af78c5c65c606d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:54.761969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb4d57-0309-4a1b-8a1c-b51c63731891 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:54.763384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-b3eb4d57-0309-4a1b-8a1c-b51c63731891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:54.776879 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb4d57-0309-4a1b-8a1c-b51c63731891 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:54.834409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30729ca1-a2b3-4feb-a7a8-fa3e01994f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:54.835322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30729ca1-a2b3-4feb-a7a8-fa3e01994f42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:54.835611 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30729ca1-a2b3-4feb-a7a8-fa3e01994f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:54.835915 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30729ca1-a2b3-4feb-a7a8-fa3e01994f42 admin admin] PATH_INFO: `/v3/users/ee42a33caed34a14964801a91a06ece2` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:54.837093 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30729ca1-a2b3-4feb-a7a8-fa3e01994f42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee42a33caed34a14964801a91a06ece2)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:54.839139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30729ca1-a2b3-4feb-a7a8-fa3e01994f42 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:54.887244 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-30729ca1-a2b3-4feb-a7a8-fa3e01994f42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee42a33caed34a14964801a91a06ece2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:54.894623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-30729ca1-a2b3-4feb-a7a8-fa3e01994f42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee42a33caed34a14964801a91a06ece2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:54.902560 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-30729ca1-a2b3-4feb-a7a8-fa3e01994f42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee42a33caed34a14964801a91a06ece2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:54.903349 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-30729ca1-a2b3-4feb-a7a8-fa3e01994f42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee42a33caed34a14964801a91a06ece2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:54.914976 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-30729ca1-a2b3-4feb-a7a8-fa3e01994f42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee42a33caed34a14964801a91a06ece2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:54.921759 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1277/2558] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:44:54 2022] DELETE /identity/v3/users/ee42a33caed34a14964801a91a06ece2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:55.187974 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352db591-506a-4374-a38a-174a9f63ab2c admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:55.189481 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-352db591-506a-4374-a38a-174a9f63ab2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:55.203136 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352db591-506a-4374-a38a-174a9f63ab2c admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:55.270190 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d7e56-4367-4844-898e-1b72c8dca709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:55.271040 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d7e56-4367-4844-898e-1b72c8dca709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:55.271316 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d7e56-4367-4844-898e-1b72c8dca709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:55.271636 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d7e56-4367-4844-898e-1b72c8dca709 admin admin] PATH_INFO: `/v3/projects/8ba286a5d80b4a048ee93e18202722f0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:55.272806 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d7e56-4367-4844-898e-1b72c8dca709 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ba286a5d80b4a048ee93e18202722f0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:55.275310 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d7e56-4367-4844-898e-1b72c8dca709 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:55.308403 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:55.323765 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-db2d7e56-4367-4844-898e-1b72c8dca709 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ba286a5d80b4a048ee93e18202722f0', 'request_id': 'req-db2d7e56-4367-4844-898e-1b72c8dca709'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:55.330913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-db2d7e56-4367-4844-898e-1b72c8dca709 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ba286a5d80b4a048ee93e18202722f0', 'request_id': 'req-db2d7e56-4367-4844-898e-1b72c8dca709'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:55.336479 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-db2d7e56-4367-4844-898e-1b72c8dca709 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ba286a5d80b4a048ee93e18202722f0', 'request_id': 'req-db2d7e56-4367-4844-898e-1b72c8dca709'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:55.348332 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-db2d7e56-4367-4844-898e-1b72c8dca709 admin admin] The token cache is being invalidate because project 8ba286a5d80b4a048ee93e18202722f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:44:55.348615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-db2d7e56-4367-4844-898e-1b72c8dca709 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:55.349253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-db2d7e56-4367-4844-898e-1b72c8dca709 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:44:55.350789 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1282/2559] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:44:55 2022] DELETE /identity/v3/projects/8ba286a5d80b4a048ee93e18202722f0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:44:55.355895 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a55e7-8fef-4de1-82e5-ca4ab879f2e2 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:55.356259 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a55e7-8fef-4de1-82e5-ca4ab879f2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:55.356618 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a55e7-8fef-4de1-82e5-ca4ab879f2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:55.375129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-585a55e7-8fef-4de1-82e5-ca4ab879f2e2 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:55.376405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-585a55e7-8fef-4de1-82e5-ca4ab879f2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:55.447867 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1278/2560] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:55.455588 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf5394-9256-4bb3-80d8-e7b263c685fc None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:55.455908 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf5394-9256-4bb3-80d8-e7b263c685fc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:55.456184 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf5394-9256-4bb3-80d8-e7b263c685fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:55.474792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2dcf5394-9256-4bb3-80d8-e7b263c685fc None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:55.476041 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2dcf5394-9256-4bb3-80d8-e7b263c685fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:55.483971 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1283/2561] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:55.488090 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a55e7-8fef-4de1-82e5-ca4ab879f2e2 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:55.497772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a55e7-8fef-4de1-82e5-ca4ab879f2e2 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:55.502401 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf5f11-a3e8-4bfe-a702-f20698326ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:55.503044 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf5f11-a3e8-4bfe-a702-f20698326ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:55.503327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf5f11-a3e8-4bfe-a702-f20698326ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:55.503605 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf5f11-a3e8-4bfe-a702-f20698326ab8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:55.504434 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adf5f11-a3e8-4bfe-a702-f20698326ab8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:44:55.504742 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adf5f11-a3e8-4bfe-a702-f20698326ab8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:55.506327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adf5f11-a3e8-4bfe-a702-f20698326ab8 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:55.518479 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1279/2562] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:44:55.522537 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf5394-9256-4bb3-80d8-e7b263c685fc None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:55.532170 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf5394-9256-4bb3-80d8-e7b263c685fc None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:55.536769 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de91cd-4695-450d-93d3-fb37789a66c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:55.537398 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de91cd-4695-450d-93d3-fb37789a66c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:55.537676 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de91cd-4695-450d-93d3-fb37789a66c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:55.537946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de91cd-4695-450d-93d3-fb37789a66c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:55.538617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18de91cd-4695-450d-93d3-fb37789a66c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:55.540743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18de91cd-4695-450d-93d3-fb37789a66c6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:55.565921 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1284/2563] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:44:55 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:44:55.570013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf5f11-a3e8-4bfe-a702-f20698326ab8 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:55.575517 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:55.582196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf5f11-a3e8-4bfe-a702-f20698326ab8 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:55.642263 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c6abc-e458-400c-92b3-f5db97fb46fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:55.642948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c6abc-e458-400c-92b3-f5db97fb46fe admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:55.643236 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c6abc-e458-400c-92b3-f5db97fb46fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:55.643513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c6abc-e458-400c-92b3-f5db97fb46fe admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:55.644247 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c6abc-e458-400c-92b3-f5db97fb46fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:55.646186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c6abc-e458-400c-92b3-f5db97fb46fe admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:55.666734 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1280/2564] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:44:55 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:44:55.670860 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de91cd-4695-450d-93d3-fb37789a66c6 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:55.681444 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de91cd-4695-450d-93d3-fb37789a66c6 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:55.685991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df677dc1-2558-425c-9f71-ebf6d6ac669d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:55.686639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df677dc1-2558-425c-9f71-ebf6d6ac669d admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:55.686922 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df677dc1-2558-425c-9f71-ebf6d6ac669d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:55.687198 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df677dc1-2558-425c-9f71-ebf6d6ac669d admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:55.687958 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df677dc1-2558-425c-9f71-ebf6d6ac669d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:55.689540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df677dc1-2558-425c-9f71-ebf6d6ac669d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:55.700833 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1285/2565] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:55.704578 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c6abc-e458-400c-92b3-f5db97fb46fe admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:55.714999 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c6abc-e458-400c-92b3-f5db97fb46fe admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:55.719619 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd24f3-6a4e-4023-8763-65220be6f5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:55.720304 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd24f3-6a4e-4023-8763-65220be6f5c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:55.720586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd24f3-6a4e-4023-8763-65220be6f5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:55.720863 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd24f3-6a4e-4023-8763-65220be6f5c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:55.721575 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd24f3-6a4e-4023-8763-65220be6f5c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:55.723147 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd24f3-6a4e-4023-8763-65220be6f5c5 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:55.733678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1281/2566] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:44:55.737789 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df677dc1-2558-425c-9f71-ebf6d6ac669d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:55.747768 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df677dc1-2558-425c-9f71-ebf6d6ac669d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:55.752410 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a5875-11c5-4b73-a35a-f67ff40504fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:55.753272 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a5875-11c5-4b73-a35a-f67ff40504fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:55.753554 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a5875-11c5-4b73-a35a-f67ff40504fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:55.753825 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a5875-11c5-4b73-a35a-f67ff40504fa admin admin] PATH_INFO: `/v3/projects/d93c9242f08447aaa597423812d5904d/users/63d02effcaa64a2580cdd1613e7ff1d8/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:55.768367 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a5875-11c5-4b73-a35a-f67ff40504fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d93c9242f08447aaa597423812d5904d, user_id=63d02effcaa64a2580cdd1613e7ff1d8, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:55.771018 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a5875-11c5-4b73-a35a-f67ff40504fa admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:55.784054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1286/2567] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:44:55 2022] PUT /identity/v3/projects/d93c9242f08447aaa597423812d5904d/users/63d02effcaa64a2580cdd1613e7ff1d8/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:44:55.789157 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a5162-606a-4e93-980c-1b4227f33559 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:55.789629 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a5162-606a-4e93-980c-1b4227f33559 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:55.789731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a5162-606a-4e93-980c-1b4227f33559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:55.808339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-3d9a5162-606a-4e93-980c-1b4227f33559 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:55.809543 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-3d9a5162-606a-4e93-980c-1b4227f33559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:55.867592 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1282/2568] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:55.877186 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a5875-11c5-4b73-a35a-f67ff40504fa admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:55.878896 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b45a5875-11c5-4b73-a35a-f67ff40504fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:55.938346 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a5875-11c5-4b73-a35a-f67ff40504fa admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:55.942019 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8daa55-8fd0-4ed4-a406-0d3d1fe5dc2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:55.942680 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8daa55-8fd0-4ed4-a406-0d3d1fe5dc2e service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:55.942944 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8daa55-8fd0-4ed4-a406-0d3d1fe5dc2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:55.943217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8daa55-8fd0-4ed4-a406-0d3d1fe5dc2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:55.952472 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8daa55-8fd0-4ed4-a406-0d3d1fe5dc2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:55.953855 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8daa55-8fd0-4ed4-a406-0d3d1fe5dc2e service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:55.963212 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1287/2569] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:44:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:44:56.581341 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba68fa3-ba7f-4d08-9bb6-bce84eb5959d None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:56.581868 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba68fa3-ba7f-4d08-9bb6-bce84eb5959d None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:56.582177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba68fa3-ba7f-4d08-9bb6-bce84eb5959d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:56.610579 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-1ba68fa3-ba7f-4d08-9bb6-bce84eb5959d None None] MFA Rules not processed for user `63d02effcaa64a2580cdd1613e7ff1d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:56.611827 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-1ba68fa3-ba7f-4d08-9bb6-bce84eb5959d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:56.679457 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1283/2570] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:56.690559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8daa55-8fd0-4ed4-a406-0d3d1fe5dc2e service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:56.692063 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5d8daa55-8fd0-4ed4-a406-0d3d1fe5dc2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:56.750551 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8daa55-8fd0-4ed4-a406-0d3d1fe5dc2e service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:56.754347 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a46356-a682-4daa-8916-eb944b86474e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:56.754991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a46356-a682-4daa-8916-eb944b86474e service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:56.755274 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a46356-a682-4daa-8916-eb944b86474e service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:56.755604 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a46356-a682-4daa-8916-eb944b86474e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:56.765633 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a46356-a682-4daa-8916-eb944b86474e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:56.766999 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a46356-a682-4daa-8916-eb944b86474e service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:56.771543 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:44:56.777048 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1288/2571] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:44:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:44:58.117570 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0922f9-f51f-4a7a-bfac-0e121c45d95e None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:58.118135 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0922f9-f51f-4a7a-bfac-0e121c45d95e None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:58.118135 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0922f9-f51f-4a7a-bfac-0e121c45d95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:58.136840 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-2f0922f9-f51f-4a7a-bfac-0e121c45d95e None None] MFA Rules not processed for user `63d02effcaa64a2580cdd1613e7ff1d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:44:58.138035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-2f0922f9-f51f-4a7a-bfac-0e121c45d95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:58.145102 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1284/2572] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:44:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:44:59.191833 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a46356-a682-4daa-8916-eb944b86474e service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:44:59.203672 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a46356-a682-4daa-8916-eb944b86474e service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:44:59.215818 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff10e2a-9dea-4484-bce2-193a78cef128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:44:59.216507 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff10e2a-9dea-4484-bce2-193a78cef128 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:44:59.216889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff10e2a-9dea-4484-bce2-193a78cef128 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:44:59.217237 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff10e2a-9dea-4484-bce2-193a78cef128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:44:59.219152 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-7ff10e2a-9dea-4484-bce2-193a78cef128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:44:59.278345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff10e2a-9dea-4484-bce2-193a78cef128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:44:59.279765 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff10e2a-9dea-4484-bce2-193a78cef128 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:44:59.288959 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1289/2573] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:44:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:45:05.507049 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0922f9-f51f-4a7a-bfac-0e121c45d95e None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:05.513175 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:05.518127 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0922f9-f51f-4a7a-bfac-0e121c45d95e None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:05.522501 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c991bb-a6fb-4fea-9469-e27fb3334dd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:05.523161 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c991bb-a6fb-4fea-9469-e27fb3334dd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:05.523451 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c991bb-a6fb-4fea-9469-e27fb3334dd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:05.523731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c991bb-a6fb-4fea-9469-e27fb3334dd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:05.526150 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-73c991bb-a6fb-4fea-9469-e27fb3334dd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:05.538061 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c991bb-a6fb-4fea-9469-e27fb3334dd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:05.539469 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c991bb-a6fb-4fea-9469-e27fb3334dd2 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:05.549647 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1285/2574] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:45:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:45:09.284676 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=28 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:12.618515 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff10e2a-9dea-4484-bce2-193a78cef128 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:12.621287 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-7ff10e2a-9dea-4484-bce2-193a78cef128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:12.644219 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff10e2a-9dea-4484-bce2-193a78cef128 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:12.660781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6f0d0-53a6-4edc-85bf-06a56b83ac4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:12.660781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6f0d0-53a6-4edc-85bf-06a56b83ac4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:12.660781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6f0d0-53a6-4edc-85bf-06a56b83ac4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:12.660781 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6f0d0-53a6-4edc-85bf-06a56b83ac4f admin admin] PATH_INFO: `/v3/users/6d09fe912b4c4cab81b00df801f69680` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:12.661090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a6f0d0-53a6-4edc-85bf-06a56b83ac4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d09fe912b4c4cab81b00df801f69680)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:12.661090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a6f0d0-53a6-4edc-85bf-06a56b83ac4f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:12.711743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-07a6f0d0-53a6-4edc-85bf-06a56b83ac4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d09fe912b4c4cab81b00df801f69680'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:12.719271 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-07a6f0d0-53a6-4edc-85bf-06a56b83ac4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d09fe912b4c4cab81b00df801f69680'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:12.719980 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-07a6f0d0-53a6-4edc-85bf-06a56b83ac4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d09fe912b4c4cab81b00df801f69680'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:12.720601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-07a6f0d0-53a6-4edc-85bf-06a56b83ac4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d09fe912b4c4cab81b00df801f69680'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:12.734477 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-07a6f0d0-53a6-4edc-85bf-06a56b83ac4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d09fe912b4c4cab81b00df801f69680'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:12.759183 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1290/2575] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:45:12 2022] DELETE /identity/v3/users/6d09fe912b4c4cab81b00df801f69680 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:12.978299 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c991bb-a6fb-4fea-9469-e27fb3334dd2 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:12.993013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c991bb-a6fb-4fea-9469-e27fb3334dd2 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:13.075975 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaef9f4-d510-44c1-bf6a-fa6b0f8bdaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:13.077135 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaef9f4-d510-44c1-bf6a-fa6b0f8bdaee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:13.077446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaef9f4-d510-44c1-bf6a-fa6b0f8bdaee admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:13.077876 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaef9f4-d510-44c1-bf6a-fa6b0f8bdaee admin admin] PATH_INFO: `/v3/projects/ef2496096b374f399606bff9a08b65eb` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:13.079074 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaef9f4-d510-44c1-bf6a-fa6b0f8bdaee admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef2496096b374f399606bff9a08b65eb)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:13.081035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaef9f4-d510-44c1-bf6a-fa6b0f8bdaee admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:13.148437 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-fdaef9f4-d510-44c1-bf6a-fa6b0f8bdaee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef2496096b374f399606bff9a08b65eb', 'request_id': 'req-fdaef9f4-d510-44c1-bf6a-fa6b0f8bdaee'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:13.154199 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-fdaef9f4-d510-44c1-bf6a-fa6b0f8bdaee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef2496096b374f399606bff9a08b65eb', 'request_id': 'req-fdaef9f4-d510-44c1-bf6a-fa6b0f8bdaee'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:13.162291 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-fdaef9f4-d510-44c1-bf6a-fa6b0f8bdaee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef2496096b374f399606bff9a08b65eb', 'request_id': 'req-fdaef9f4-d510-44c1-bf6a-fa6b0f8bdaee'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:13.178631 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-fdaef9f4-d510-44c1-bf6a-fa6b0f8bdaee admin admin] The token cache is being invalidate because project ef2496096b374f399606bff9a08b65eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:45:13.178631 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-fdaef9f4-d510-44c1-bf6a-fa6b0f8bdaee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:13.179726 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-fdaef9f4-d510-44c1-bf6a-fa6b0f8bdaee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:13.182066 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1286/2576] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:45:12 2022] DELETE /identity/v3/projects/ef2496096b374f399606bff9a08b65eb => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:13.188633 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6f0d0-53a6-4edc-85bf-06a56b83ac4f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:13.190061 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-07a6f0d0-53a6-4edc-85bf-06a56b83ac4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:13.203151 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6f0d0-53a6-4edc-85bf-06a56b83ac4f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:13.278937 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347df853-272c-48fa-a3fb-bdc4de3ab730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:13.278937 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347df853-272c-48fa-a3fb-bdc4de3ab730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:13.278937 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347df853-272c-48fa-a3fb-bdc4de3ab730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:13.278937 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347df853-272c-48fa-a3fb-bdc4de3ab730 admin admin] PATH_INFO: `/v3/users/35e8344f1cd44395a4b90b273d19259b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:13.283161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347df853-272c-48fa-a3fb-bdc4de3ab730 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35e8344f1cd44395a4b90b273d19259b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:13.285216 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347df853-272c-48fa-a3fb-bdc4de3ab730 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:13.347611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-347df853-272c-48fa-a3fb-bdc4de3ab730 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35e8344f1cd44395a4b90b273d19259b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:13.358917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-347df853-272c-48fa-a3fb-bdc4de3ab730 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35e8344f1cd44395a4b90b273d19259b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:13.359907 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-347df853-272c-48fa-a3fb-bdc4de3ab730 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35e8344f1cd44395a4b90b273d19259b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:13.360687 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-347df853-272c-48fa-a3fb-bdc4de3ab730 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35e8344f1cd44395a4b90b273d19259b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:13.375506 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-347df853-272c-48fa-a3fb-bdc4de3ab730 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35e8344f1cd44395a4b90b273d19259b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:13.380277 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:13.402819 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1291/2577] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:45:13 2022] DELETE /identity/v3/users/35e8344f1cd44395a4b90b273d19259b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:45:13.686052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaef9f4-d510-44c1-bf6a-fa6b0f8bdaee admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:13.687657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-fdaef9f4-d510-44c1-bf6a-fa6b0f8bdaee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:13.708665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaef9f4-d510-44c1-bf6a-fa6b0f8bdaee admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:13.787144 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd232d8e-5d86-4ca0-8515-a19cf757abf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:13.788907 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd232d8e-5d86-4ca0-8515-a19cf757abf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:13.788907 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd232d8e-5d86-4ca0-8515-a19cf757abf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:13.788907 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd232d8e-5d86-4ca0-8515-a19cf757abf3 admin admin] PATH_INFO: `/v3/projects/1679763b60fd4d1fb461e9881df094c0` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:13.790744 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd232d8e-5d86-4ca0-8515-a19cf757abf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1679763b60fd4d1fb461e9881df094c0)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:13.792773 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd232d8e-5d86-4ca0-8515-a19cf757abf3 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:13.820709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:13.860620 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-cd232d8e-5d86-4ca0-8515-a19cf757abf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1679763b60fd4d1fb461e9881df094c0', 'request_id': 'req-cd232d8e-5d86-4ca0-8515-a19cf757abf3'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:13.867318 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-cd232d8e-5d86-4ca0-8515-a19cf757abf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1679763b60fd4d1fb461e9881df094c0', 'request_id': 'req-cd232d8e-5d86-4ca0-8515-a19cf757abf3'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:13.876240 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-cd232d8e-5d86-4ca0-8515-a19cf757abf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1679763b60fd4d1fb461e9881df094c0', 'request_id': 'req-cd232d8e-5d86-4ca0-8515-a19cf757abf3'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:13.890081 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-cd232d8e-5d86-4ca0-8515-a19cf757abf3 admin admin] The token cache is being invalidate because project 1679763b60fd4d1fb461e9881df094c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:45:13.890460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-cd232d8e-5d86-4ca0-8515-a19cf757abf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:13.891121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-cd232d8e-5d86-4ca0-8515-a19cf757abf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:13.892777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1287/2578] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:45:13 2022] DELETE /identity/v3/projects/1679763b60fd4d1fb461e9881df094c0 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:16.483738 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347df853-272c-48fa-a3fb-bdc4de3ab730 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:16.485162 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-347df853-272c-48fa-a3fb-bdc4de3ab730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:16.541495 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347df853-272c-48fa-a3fb-bdc4de3ab730 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:16.593326 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e97ae2-ea2e-4e1d-b65a-f04126e0b42b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:16.593971 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e97ae2-ea2e-4e1d-b65a-f04126e0b42b service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:16.594248 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e97ae2-ea2e-4e1d-b65a-f04126e0b42b service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:16.594517 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e97ae2-ea2e-4e1d-b65a-f04126e0b42b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:16.596337 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-08e97ae2-ea2e-4e1d-b65a-f04126e0b42b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:16.661706 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e97ae2-ea2e-4e1d-b65a-f04126e0b42b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:16.663143 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e97ae2-ea2e-4e1d-b65a-f04126e0b42b service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:16.673235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1292/2579] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:45:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:45:19.611654 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd232d8e-5d86-4ca0-8515-a19cf757abf3 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:19.613202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-cd232d8e-5d86-4ca0-8515-a19cf757abf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:19.626079 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd232d8e-5d86-4ca0-8515-a19cf757abf3 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:19.630936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f196837-0d84-43e9-9be0-094b42184f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:19.631838 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f196837-0d84-43e9-9be0-094b42184f58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:19.632181 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f196837-0d84-43e9-9be0-094b42184f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:19.632446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f196837-0d84-43e9-9be0-094b42184f58 admin admin] PATH_INFO: `/v3/users/ea1482ba53554f71865cea46ffbac23c` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:19.633757 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f196837-0d84-43e9-9be0-094b42184f58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea1482ba53554f71865cea46ffbac23c)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:19.635690 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f196837-0d84-43e9-9be0-094b42184f58 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:19.688461 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9f196837-0d84-43e9-9be0-094b42184f58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea1482ba53554f71865cea46ffbac23c'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:19.696684 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9f196837-0d84-43e9-9be0-094b42184f58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea1482ba53554f71865cea46ffbac23c'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:19.709269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9f196837-0d84-43e9-9be0-094b42184f58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea1482ba53554f71865cea46ffbac23c'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:19.709992 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9f196837-0d84-43e9-9be0-094b42184f58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea1482ba53554f71865cea46ffbac23c'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:19.724506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9f196837-0d84-43e9-9be0-094b42184f58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea1482ba53554f71865cea46ffbac23c'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:19.734554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1288/2580] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:45:19 2022] DELETE /identity/v3/users/ea1482ba53554f71865cea46ffbac23c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:19.922256 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e97ae2-ea2e-4e1d-b65a-f04126e0b42b service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:19.933049 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e97ae2-ea2e-4e1d-b65a-f04126e0b42b service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:19.987832 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:19.995064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f2918-e8b7-42eb-b514-acaebcf077c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:19.996098 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f2918-e8b7-42eb-b514-acaebcf077c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:19.996615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f2918-e8b7-42eb-b514-acaebcf077c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:19.997078 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f2918-e8b7-42eb-b514-acaebcf077c8 admin admin] PATH_INFO: `/v3/projects/4905170c438f456182ba201316a64029` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:19.998507 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f2918-e8b7-42eb-b514-acaebcf077c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4905170c438f456182ba201316a64029)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:20.000483 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f2918-e8b7-42eb-b514-acaebcf077c8 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:20.050228 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-409f2918-e8b7-42eb-b514-acaebcf077c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4905170c438f456182ba201316a64029', 'request_id': 'req-409f2918-e8b7-42eb-b514-acaebcf077c8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:20.057483 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-409f2918-e8b7-42eb-b514-acaebcf077c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4905170c438f456182ba201316a64029', 'request_id': 'req-409f2918-e8b7-42eb-b514-acaebcf077c8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:20.062942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-409f2918-e8b7-42eb-b514-acaebcf077c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4905170c438f456182ba201316a64029', 'request_id': 'req-409f2918-e8b7-42eb-b514-acaebcf077c8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:20.074502 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-409f2918-e8b7-42eb-b514-acaebcf077c8 admin admin] The token cache is being invalidate because project 4905170c438f456182ba201316a64029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:45:20.074790 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-409f2918-e8b7-42eb-b514-acaebcf077c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:20.075418 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-409f2918-e8b7-42eb-b514-acaebcf077c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:20.076998 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1293/2581] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:45:19 2022] DELETE /identity/v3/projects/4905170c438f456182ba201316a64029 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:20.081905 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21a598-a787-42bd-be84-736cdcc89a6c None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:20.082230 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21a598-a787-42bd-be84-736cdcc89a6c None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:20.082512 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21a598-a787-42bd-be84-736cdcc89a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:20.100501 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-be21a598-a787-42bd-be84-736cdcc89a6c None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:45:20.101735 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-be21a598-a787-42bd-be84-736cdcc89a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:20.163645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1289/2582] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:45:20.171412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72b95c-9027-497f-a4fa-ae1ee52fc377 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:20.171735 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72b95c-9027-497f-a4fa-ae1ee52fc377 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:20.172046 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72b95c-9027-497f-a4fa-ae1ee52fc377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:20.190878 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-dd72b95c-9027-497f-a4fa-ae1ee52fc377 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:45:20.192131 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-dd72b95c-9027-497f-a4fa-ae1ee52fc377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:20.200730 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1294/2583] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:45:20.204913 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21a598-a787-42bd-be84-736cdcc89a6c None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:20.215247 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21a598-a787-42bd-be84-736cdcc89a6c None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:20.219743 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e2137-746c-463a-b7a0-27bade6822e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:20.220416 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e2137-746c-463a-b7a0-27bade6822e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:20.220696 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e2137-746c-463a-b7a0-27bade6822e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:20.220968 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e2137-746c-463a-b7a0-27bade6822e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:20.221683 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e2137-746c-463a-b7a0-27bade6822e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:45:20.221983 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e2137-746c-463a-b7a0-27bade6822e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:20.223566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e2137-746c-463a-b7a0-27bade6822e9 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:20.235151 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1290/2584] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:45:20.239298 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72b95c-9027-497f-a4fa-ae1ee52fc377 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:20.248625 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72b95c-9027-497f-a4fa-ae1ee52fc377 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:20.253253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631ddbe-2732-4bcb-993c-0e09b0eb687f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:20.253897 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631ddbe-2732-4bcb-993c-0e09b0eb687f admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:20.254179 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631ddbe-2732-4bcb-993c-0e09b0eb687f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:20.254453 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631ddbe-2732-4bcb-993c-0e09b0eb687f admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:20.255127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631ddbe-2732-4bcb-993c-0e09b0eb687f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:20.257041 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631ddbe-2732-4bcb-993c-0e09b0eb687f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:20.282949 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1295/2585] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:45:20 2022] POST /identity/v3/projects => generated 355 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:45:20.287130 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e2137-746c-463a-b7a0-27bade6822e9 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:20.297524 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e2137-746c-463a-b7a0-27bade6822e9 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:20.305951 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:20.357921 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d750c90-663a-4112-bf51-aaefdbc2e0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:20.358586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d750c90-663a-4112-bf51-aaefdbc2e0f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:20.358866 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d750c90-663a-4112-bf51-aaefdbc2e0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:20.359144 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d750c90-663a-4112-bf51-aaefdbc2e0f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:20.359842 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d750c90-663a-4112-bf51-aaefdbc2e0f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:20.361745 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d750c90-663a-4112-bf51-aaefdbc2e0f8 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:20.382482 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1291/2586] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:45:20 2022] POST /identity/v3/users => generated 278 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:45:20.387324 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631ddbe-2732-4bcb-993c-0e09b0eb687f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:20.397811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631ddbe-2732-4bcb-993c-0e09b0eb687f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:20.402697 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82c161-d7fd-40f8-8f73-e42fe8d8e6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:20.403339 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82c161-d7fd-40f8-8f73-e42fe8d8e6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:20.403617 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82c161-d7fd-40f8-8f73-e42fe8d8e6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:20.403929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82c161-d7fd-40f8-8f73-e42fe8d8e6b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:20.404642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82c161-d7fd-40f8-8f73-e42fe8d8e6b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:20.406208 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82c161-d7fd-40f8-8f73-e42fe8d8e6b9 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:20.416505 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1296/2587] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:45:20.420411 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d750c90-663a-4112-bf51-aaefdbc2e0f8 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:20.430800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d750c90-663a-4112-bf51-aaefdbc2e0f8 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:20.435515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b6c00-9662-4fca-97c7-d0b1742ee6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:20.436190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b6c00-9662-4fca-97c7-d0b1742ee6ca admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:20.436475 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b6c00-9662-4fca-97c7-d0b1742ee6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:20.436748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b6c00-9662-4fca-97c7-d0b1742ee6ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:20.437463 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b6c00-9662-4fca-97c7-d0b1742ee6ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:20.439033 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b6c00-9662-4fca-97c7-d0b1742ee6ca admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:20.449472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1292/2588] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:45:20.453553 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82c161-d7fd-40f8-8f73-e42fe8d8e6b9 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:20.463662 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82c161-d7fd-40f8-8f73-e42fe8d8e6b9 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:20.468296 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162bedf-2abc-453c-a454-750e1b8af08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:20.469191 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162bedf-2abc-453c-a454-750e1b8af08f admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:20.469476 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162bedf-2abc-453c-a454-750e1b8af08f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:20.469753 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162bedf-2abc-453c-a454-750e1b8af08f admin admin] PATH_INFO: `/v3/projects/6bb9a18a245344818189853eca17f337/users/775046221a56418abe8a90a72ad2d943/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:20.484590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162bedf-2abc-453c-a454-750e1b8af08f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bb9a18a245344818189853eca17f337, user_id=775046221a56418abe8a90a72ad2d943, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:20.487223 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162bedf-2abc-453c-a454-750e1b8af08f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:20.501090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1297/2589] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:45:20 2022] PUT /identity/v3/projects/6bb9a18a245344818189853eca17f337/users/775046221a56418abe8a90a72ad2d943/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:20.511720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fbe0b-6a94-448e-a5a9-1b571ff9853d None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:20.512039 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fbe0b-6a94-448e-a5a9-1b571ff9853d None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:20.512315 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fbe0b-6a94-448e-a5a9-1b571ff9853d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:20.539658 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-cb1fbe0b-6a94-448e-a5a9-1b571ff9853d None None] MFA Rules not processed for user `775046221a56418abe8a90a72ad2d943`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:45:20.540887 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-cb1fbe0b-6a94-448e-a5a9-1b571ff9853d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:20.606018 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1293/2590] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:45:20 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:45:20.610465 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162bedf-2abc-453c-a454-750e1b8af08f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:20.671364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162bedf-2abc-453c-a454-750e1b8af08f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:20.675973 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae214a-52de-4e7a-ac28-a1c3d010265a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:20.676622 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae214a-52de-4e7a-ac28-a1c3d010265a admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:20.676904 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae214a-52de-4e7a-ac28-a1c3d010265a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:20.677180 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae214a-52de-4e7a-ac28-a1c3d010265a admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:20.677872 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae214a-52de-4e7a-ac28-a1c3d010265a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:20.680413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae214a-52de-4e7a-ac28-a1c3d010265a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:20.705178 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1298/2591] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:45:20 2022] POST /identity/v3/projects => generated 357 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:45:20.709252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1fbe0b-6a94-448e-a5a9-1b571ff9853d None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:20.720004 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1fbe0b-6a94-448e-a5a9-1b571ff9853d None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:20.779512 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5f151-4a4d-4d0d-a7aa-ea8d4624138c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:20.780274 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5f151-4a4d-4d0d-a7aa-ea8d4624138c admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:20.780561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5f151-4a4d-4d0d-a7aa-ea8d4624138c admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:20.781171 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5f151-4a4d-4d0d-a7aa-ea8d4624138c admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:20.781540 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5f151-4a4d-4d0d-a7aa-ea8d4624138c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:20.783519 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5f151-4a4d-4d0d-a7aa-ea8d4624138c admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:20.805162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1294/2592] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:45:20 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:45:20.809195 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae214a-52de-4e7a-ac28-a1c3d010265a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:20.819878 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae214a-52de-4e7a-ac28-a1c3d010265a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:20.824939 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae5c3f-8cfd-4f96-be65-a62a080dacb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:20.825601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae5c3f-8cfd-4f96-be65-a62a080dacb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:20.825907 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae5c3f-8cfd-4f96-be65-a62a080dacb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:20.826184 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae5c3f-8cfd-4f96-be65-a62a080dacb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:20.826979 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae5c3f-8cfd-4f96-be65-a62a080dacb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:20.828657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae5c3f-8cfd-4f96-be65-a62a080dacb1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:20.830815 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:20.841560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1299/2593] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:45:20.846478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5f151-4a4d-4d0d-a7aa-ea8d4624138c admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:20.857451 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5f151-4a4d-4d0d-a7aa-ea8d4624138c admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:20.862136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc4951-eafc-4c17-9ad9-8c726e5e3392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:20.863020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc4951-eafc-4c17-9ad9-8c726e5e3392 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:20.863305 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc4951-eafc-4c17-9ad9-8c726e5e3392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:20.863581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc4951-eafc-4c17-9ad9-8c726e5e3392 admin admin] PATH_INFO: `/v3/projects/7e513a8c320d47bbab0d40a6f2bd68c9/users/4d25d56916144ca3bb937cb0f8e9c0af/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:20.866892 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:20.881257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc4951-eafc-4c17-9ad9-8c726e5e3392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e513a8c320d47bbab0d40a6f2bd68c9, user_id=4d25d56916144ca3bb937cb0f8e9c0af, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:20.884582 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc4951-eafc-4c17-9ad9-8c726e5e3392 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:20.897981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1295/2594] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:45:20 2022] PUT /identity/v3/projects/7e513a8c320d47bbab0d40a6f2bd68c9/users/4d25d56916144ca3bb937cb0f8e9c0af/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:20.902678 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae5c3f-8cfd-4f96-be65-a62a080dacb1 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:20.913392 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae5c3f-8cfd-4f96-be65-a62a080dacb1 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:20.975739 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f5a30-0902-4b5d-b563-559e68c759e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:20.976941 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f5a30-0902-4b5d-b563-559e68c759e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:20.977889 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f5a30-0902-4b5d-b563-559e68c759e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:20.978169 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f5a30-0902-4b5d-b563-559e68c759e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:20.978923 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f5a30-0902-4b5d-b563-559e68c759e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:20.980547 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f5a30-0902-4b5d-b563-559e68c759e2 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:20.994070 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1300/2595] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:45:20.995225 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc4951-eafc-4c17-9ad9-8c726e5e3392 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:21.005456 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc4951-eafc-4c17-9ad9-8c726e5e3392 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:21.011195 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1b6cc-1c5f-43ed-852b-19c536be5695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:21.012169 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1b6cc-1c5f-43ed-852b-19c536be5695 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:21.012431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1b6cc-1c5f-43ed-852b-19c536be5695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:21.012777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1b6cc-1c5f-43ed-852b-19c536be5695 admin admin] PATH_INFO: `/v3/projects/7e513a8c320d47bbab0d40a6f2bd68c9/users/4d25d56916144ca3bb937cb0f8e9c0af/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:21.014917 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1b6cc-1c5f-43ed-852b-19c536be5695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e513a8c320d47bbab0d40a6f2bd68c9, user_id=4d25d56916144ca3bb937cb0f8e9c0af, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:21.018136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1b6cc-1c5f-43ed-852b-19c536be5695 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:21.029924 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1296/2596] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:45:20 2022] PUT /identity/v3/projects/7e513a8c320d47bbab0d40a6f2bd68c9/users/4d25d56916144ca3bb937cb0f8e9c0af/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:21.042472 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980a2cf-0702-49ab-8e06-f1943a076f8b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:21.042766 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980a2cf-0702-49ab-8e06-f1943a076f8b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:21.043078 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980a2cf-0702-49ab-8e06-f1943a076f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:21.068982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-5980a2cf-0702-49ab-8e06-f1943a076f8b None None] MFA Rules not processed for user `4d25d56916144ca3bb937cb0f8e9c0af`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:45:21.070695 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5980a2cf-0702-49ab-8e06-f1943a076f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:21.137708 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1301/2597] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:45:21 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:45:21.150134 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1b6cc-1c5f-43ed-852b-19c536be5695 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:21.151814 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-d2c1b6cc-1c5f-43ed-852b-19c536be5695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:21.213452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1b6cc-1c5f-43ed-852b-19c536be5695 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:21.217277 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f112747-613e-4bd6-ae86-b4fe8615ccc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:21.217861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f112747-613e-4bd6-ae86-b4fe8615ccc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:21.218144 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f112747-613e-4bd6-ae86-b4fe8615ccc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:21.218421 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f112747-613e-4bd6-ae86-b4fe8615ccc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:21.228352 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f112747-613e-4bd6-ae86-b4fe8615ccc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:21.229780 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f112747-613e-4bd6-ae86-b4fe8615ccc9 service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:21.239130 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1297/2598] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:45:21 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:45:21.309959 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980a2cf-0702-49ab-8e06-f1943a076f8b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:21.320746 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980a2cf-0702-49ab-8e06-f1943a076f8b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:21.324696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6aede5-8589-421e-a0c9-14950450722e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:21.325344 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6aede5-8589-421e-a0c9-14950450722e service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:21.325629 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6aede5-8589-421e-a0c9-14950450722e service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:21.325903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6aede5-8589-421e-a0c9-14950450722e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:21.383419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6aede5-8589-421e-a0c9-14950450722e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:21.384892 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6aede5-8589-421e-a0c9-14950450722e service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:21.394404 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1302/2599] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:45:21 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:45:31.235003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=37,keystone:INSERT=2 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:31.390145 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:36.221684 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f112747-613e-4bd6-ae86-b4fe8615ccc9 service cinder] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:36.303156 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f112747-613e-4bd6-ae86-b4fe8615ccc9 service cinder] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:36.308745 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d32a9b-46b3-406c-af76-5c2722c6cc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:36.310561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d32a9b-46b3-406c-af76-5c2722c6cc1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:36.310848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d32a9b-46b3-406c-af76-5c2722c6cc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:36.311124 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d32a9b-46b3-406c-af76-5c2722c6cc1e admin admin] PATH_INFO: `/v3/users/775046221a56418abe8a90a72ad2d943` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:36.312336 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d32a9b-46b3-406c-af76-5c2722c6cc1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=775046221a56418abe8a90a72ad2d943)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:36.314270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d32a9b-46b3-406c-af76-5c2722c6cc1e admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:36.362184 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d7d32a9b-46b3-406c-af76-5c2722c6cc1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '775046221a56418abe8a90a72ad2d943'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:36.371274 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d7d32a9b-46b3-406c-af76-5c2722c6cc1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '775046221a56418abe8a90a72ad2d943'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:36.379400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d7d32a9b-46b3-406c-af76-5c2722c6cc1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '775046221a56418abe8a90a72ad2d943'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:36.380121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d7d32a9b-46b3-406c-af76-5c2722c6cc1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '775046221a56418abe8a90a72ad2d943'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:36.393772 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d7d32a9b-46b3-406c-af76-5c2722c6cc1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '775046221a56418abe8a90a72ad2d943'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:36.400224 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1298/2600] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:45:36 2022] DELETE /identity/v3/users/775046221a56418abe8a90a72ad2d943 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:36.404710 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52743bf4-09cf-43cb-9795-af8ebc705801 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:36.405282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52743bf4-09cf-43cb-9795-af8ebc705801 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:36.405282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52743bf4-09cf-43cb-9795-af8ebc705801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:36.422822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-52743bf4-09cf-43cb-9795-af8ebc705801 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:45:36.424049 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-52743bf4-09cf-43cb-9795-af8ebc705801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:36.483370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1303/2601] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:45:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:45:36.493567 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d32a9b-46b3-406c-af76-5c2722c6cc1e admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:36.495073 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-d7d32a9b-46b3-406c-af76-5c2722c6cc1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:36.551974 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d32a9b-46b3-406c-af76-5c2722c6cc1e admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:36.556262 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37884c7a-633a-4eb2-b18d-f4649c45bbb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:36.557186 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37884c7a-633a-4eb2-b18d-f4649c45bbb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:36.557663 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37884c7a-633a-4eb2-b18d-f4649c45bbb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:36.558166 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37884c7a-633a-4eb2-b18d-f4649c45bbb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:36.570444 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37884c7a-633a-4eb2-b18d-f4649c45bbb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:36.571883 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37884c7a-633a-4eb2-b18d-f4649c45bbb9 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:36.581270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1299/2602] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:45:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:45:37.098192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52743bf4-09cf-43cb-9795-af8ebc705801 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:37.099606 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-52743bf4-09cf-43cb-9795-af8ebc705801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:37.114424 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52743bf4-09cf-43cb-9795-af8ebc705801 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:37.120589 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628766f-9aa0-44c3-b98c-ccf631984aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:37.121502 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628766f-9aa0-44c3-b98c-ccf631984aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:37.121712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628766f-9aa0-44c3-b98c-ccf631984aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:37.121984 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628766f-9aa0-44c3-b98c-ccf631984aa2 admin admin] PATH_INFO: `/v3/projects/6bb9a18a245344818189853eca17f337` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:37.123181 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b628766f-9aa0-44c3-b98c-ccf631984aa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bb9a18a245344818189853eca17f337)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:37.125054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b628766f-9aa0-44c3-b98c-ccf631984aa2 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:37.173042 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-b628766f-9aa0-44c3-b98c-ccf631984aa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bb9a18a245344818189853eca17f337', 'request_id': 'req-b628766f-9aa0-44c3-b98c-ccf631984aa2'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:37.180533 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-b628766f-9aa0-44c3-b98c-ccf631984aa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bb9a18a245344818189853eca17f337', 'request_id': 'req-b628766f-9aa0-44c3-b98c-ccf631984aa2'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:37.186239 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-b628766f-9aa0-44c3-b98c-ccf631984aa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bb9a18a245344818189853eca17f337', 'request_id': 'req-b628766f-9aa0-44c3-b98c-ccf631984aa2'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:37.197916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-b628766f-9aa0-44c3-b98c-ccf631984aa2 admin admin] The token cache is being invalidate because project 6bb9a18a245344818189853eca17f337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:45:37.198205 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-b628766f-9aa0-44c3-b98c-ccf631984aa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:37.198858 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-b628766f-9aa0-44c3-b98c-ccf631984aa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:37.200430 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1304/2603] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:45:37 2022] DELETE /identity/v3/projects/6bb9a18a245344818189853eca17f337 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:37.204530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37884c7a-633a-4eb2-b18d-f4649c45bbb9 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:37.216337 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37884c7a-633a-4eb2-b18d-f4649c45bbb9 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:37.235391 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:37.275632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc259e-0f38-4bd8-a0ef-3d5c9c3c7b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:37.276531 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc259e-0f38-4bd8-a0ef-3d5c9c3c7b9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:37.276818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc259e-0f38-4bd8-a0ef-3d5c9c3c7b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:37.277097 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc259e-0f38-4bd8-a0ef-3d5c9c3c7b9f admin admin] PATH_INFO: `/v3/users/4d25d56916144ca3bb937cb0f8e9c0af` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:37.278273 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc259e-0f38-4bd8-a0ef-3d5c9c3c7b9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d25d56916144ca3bb937cb0f8e9c0af)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:37.280221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc259e-0f38-4bd8-a0ef-3d5c9c3c7b9f admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:37.330092 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9cbc259e-0f38-4bd8-a0ef-3d5c9c3c7b9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d25d56916144ca3bb937cb0f8e9c0af'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:37.337190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9cbc259e-0f38-4bd8-a0ef-3d5c9c3c7b9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d25d56916144ca3bb937cb0f8e9c0af'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:37.344943 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9cbc259e-0f38-4bd8-a0ef-3d5c9c3c7b9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d25d56916144ca3bb937cb0f8e9c0af'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:37.345655 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9cbc259e-0f38-4bd8-a0ef-3d5c9c3c7b9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d25d56916144ca3bb937cb0f8e9c0af'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:37.358533 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9cbc259e-0f38-4bd8-a0ef-3d5c9c3c7b9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d25d56916144ca3bb937cb0f8e9c0af'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:37.364483 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1300/2604] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:45:37 2022] DELETE /identity/v3/users/4d25d56916144ca3bb937cb0f8e9c0af => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:37.640386 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628766f-9aa0-44c3-b98c-ccf631984aa2 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:37.641864 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b628766f-9aa0-44c3-b98c-ccf631984aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:37.656030 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628766f-9aa0-44c3-b98c-ccf631984aa2 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:37.714736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7defe-0576-49c3-893c-8e24cbe6d07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:37.715520 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7defe-0576-49c3-893c-8e24cbe6d07d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:37.715826 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7defe-0576-49c3-893c-8e24cbe6d07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:37.716168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7defe-0576-49c3-893c-8e24cbe6d07d admin admin] PATH_INFO: `/v3/projects/7e513a8c320d47bbab0d40a6f2bd68c9` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:37.717327 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb7defe-0576-49c3-893c-8e24cbe6d07d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e513a8c320d47bbab0d40a6f2bd68c9)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:37.719205 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb7defe-0576-49c3-893c-8e24cbe6d07d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:37.749568 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:37.767777 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3fb7defe-0576-49c3-893c-8e24cbe6d07d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e513a8c320d47bbab0d40a6f2bd68c9', 'request_id': 'req-3fb7defe-0576-49c3-893c-8e24cbe6d07d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:37.774935 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3fb7defe-0576-49c3-893c-8e24cbe6d07d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e513a8c320d47bbab0d40a6f2bd68c9', 'request_id': 'req-3fb7defe-0576-49c3-893c-8e24cbe6d07d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:37.780374 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3fb7defe-0576-49c3-893c-8e24cbe6d07d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e513a8c320d47bbab0d40a6f2bd68c9', 'request_id': 'req-3fb7defe-0576-49c3-893c-8e24cbe6d07d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:37.791117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3fb7defe-0576-49c3-893c-8e24cbe6d07d admin admin] The token cache is being invalidate because project 7e513a8c320d47bbab0d40a6f2bd68c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:45:37.791423 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3fb7defe-0576-49c3-893c-8e24cbe6d07d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:37.792058 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3fb7defe-0576-49c3-893c-8e24cbe6d07d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:37.793587 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1305/2605] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:45:37 2022] DELETE /identity/v3/projects/7e513a8c320d47bbab0d40a6f2bd68c9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:37.798507 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1e2ab-6e88-4bda-b228-9a5bc5e5e877 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:37.798798 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1e2ab-6e88-4bda-b228-9a5bc5e5e877 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:37.799078 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1e2ab-6e88-4bda-b228-9a5bc5e5e877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:37.816704 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-dfe1e2ab-6e88-4bda-b228-9a5bc5e5e877 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:45:37.817909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-dfe1e2ab-6e88-4bda-b228-9a5bc5e5e877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:37.877392 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1301/2606] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:45:37.886271 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e5ed7-f110-4a8e-8d84-b4a6774d6a1c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:37.886614 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e5ed7-f110-4a8e-8d84-b4a6774d6a1c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:37.886895 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e5ed7-f110-4a8e-8d84-b4a6774d6a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:37.905825 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-bd4e5ed7-f110-4a8e-8d84-b4a6774d6a1c None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:45:37.907033 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bd4e5ed7-f110-4a8e-8d84-b4a6774d6a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:37.915756 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1306/2607] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:45:37.920119 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1e2ab-6e88-4bda-b228-9a5bc5e5e877 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:37.930813 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1e2ab-6e88-4bda-b228-9a5bc5e5e877 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:37.935519 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749b669-aff6-4913-925b-c1961b10d809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:37.936194 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749b669-aff6-4913-925b-c1961b10d809 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:37.936477 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749b669-aff6-4913-925b-c1961b10d809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:37.936749 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749b669-aff6-4913-925b-c1961b10d809 admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:37.937464 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749b669-aff6-4913-925b-c1961b10d809 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:45:37.937766 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749b669-aff6-4913-925b-c1961b10d809 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:37.939354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749b669-aff6-4913-925b-c1961b10d809 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:37.951579 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1302/2608] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:45:37.956037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e5ed7-f110-4a8e-8d84-b4a6774d6a1c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:37.965580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e5ed7-f110-4a8e-8d84-b4a6774d6a1c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:37.970172 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f6e72-8d11-48e0-899b-23fcb50bf8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:37.970814 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f6e72-8d11-48e0-899b-23fcb50bf8fc admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:37.971098 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f6e72-8d11-48e0-899b-23fcb50bf8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:37.971372 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f6e72-8d11-48e0-899b-23fcb50bf8fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:37.972097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f6e72-8d11-48e0-899b-23fcb50bf8fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:37.974003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f6e72-8d11-48e0-899b-23fcb50bf8fc admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:37.999642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1307/2609] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:45:37 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:45:38.003886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749b669-aff6-4913-925b-c1961b10d809 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:38.014257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749b669-aff6-4913-925b-c1961b10d809 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:38.041833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:38.073261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c44c43-7cca-4cfb-b113-91152f2968f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:38.073946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c44c43-7cca-4cfb-b113-91152f2968f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:38.074224 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c44c43-7cca-4cfb-b113-91152f2968f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:38.074492 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c44c43-7cca-4cfb-b113-91152f2968f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:38.075177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c44c43-7cca-4cfb-b113-91152f2968f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:38.077122 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c44c43-7cca-4cfb-b113-91152f2968f9 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:38.096166 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1303/2610] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:45:38 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:45:38.100939 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f6e72-8d11-48e0-899b-23fcb50bf8fc admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:38.111622 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f6e72-8d11-48e0-899b-23fcb50bf8fc admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:38.117050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4312008-726e-4e6b-bff5-126d9f9d8965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:38.117706 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4312008-726e-4e6b-bff5-126d9f9d8965 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:38.117991 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4312008-726e-4e6b-bff5-126d9f9d8965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:38.118268 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4312008-726e-4e6b-bff5-126d9f9d8965 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:38.118990 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4312008-726e-4e6b-bff5-126d9f9d8965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:38.120611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4312008-726e-4e6b-bff5-126d9f9d8965 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:38.131343 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1308/2611] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:45:38.135600 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c44c43-7cca-4cfb-b113-91152f2968f9 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:38.147306 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c44c43-7cca-4cfb-b113-91152f2968f9 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:38.152538 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db76b3-8da1-4bd8-ade1-325830a3098c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:38.153243 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db76b3-8da1-4bd8-ade1-325830a3098c admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:38.153528 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db76b3-8da1-4bd8-ade1-325830a3098c admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:38.153806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db76b3-8da1-4bd8-ade1-325830a3098c admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:38.154607 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db76b3-8da1-4bd8-ade1-325830a3098c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:38.156450 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db76b3-8da1-4bd8-ade1-325830a3098c admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:38.167725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1304/2612] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:45:38.172054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4312008-726e-4e6b-bff5-126d9f9d8965 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:38.182795 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4312008-726e-4e6b-bff5-126d9f9d8965 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:38.187487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1d768-5152-47c0-bfd1-cf9d2538a2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:38.188401 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1d768-5152-47c0-bfd1-cf9d2538a2c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:38.188690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1d768-5152-47c0-bfd1-cf9d2538a2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:38.188964 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1d768-5152-47c0-bfd1-cf9d2538a2c4 admin admin] PATH_INFO: `/v3/projects/c0491e6a2f7b46c8bb03afeeb2355e75/users/6d5a63eda5e54075b7f7abeb8297357d/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:38.203397 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1d768-5152-47c0-bfd1-cf9d2538a2c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0491e6a2f7b46c8bb03afeeb2355e75, user_id=6d5a63eda5e54075b7f7abeb8297357d, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:38.206150 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1d768-5152-47c0-bfd1-cf9d2538a2c4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:38.227410 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1309/2613] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:45:38 2022] PUT /identity/v3/projects/c0491e6a2f7b46c8bb03afeeb2355e75/users/6d5a63eda5e54075b7f7abeb8297357d/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:38.232038 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab7cb5-a6f3-455c-8d15-f34f3403ef60 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:38.232331 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab7cb5-a6f3-455c-8d15-f34f3403ef60 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:38.232607 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab7cb5-a6f3-455c-8d15-f34f3403ef60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:38.261841 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-d6ab7cb5-a6f3-455c-8d15-f34f3403ef60 None None] MFA Rules not processed for user `6d5a63eda5e54075b7f7abeb8297357d`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:45:38.263312 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-d6ab7cb5-a6f3-455c-8d15-f34f3403ef60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:38.342349 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1305/2614] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:45:38 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:45:38.347105 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1d768-5152-47c0-bfd1-cf9d2538a2c4 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:38.430846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1d768-5152-47c0-bfd1-cf9d2538a2c4 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:38.435569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d0567-ff60-464f-ac63-a56cce2c4bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:38.439020 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d0567-ff60-464f-ac63-a56cce2c4bda admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:38.439510 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d0567-ff60-464f-ac63-a56cce2c4bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:38.439896 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d0567-ff60-464f-ac63-a56cce2c4bda admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:38.440639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d0567-ff60-464f-ac63-a56cce2c4bda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:38.443621 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d0567-ff60-464f-ac63-a56cce2c4bda admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:38.476683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1310/2615] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:45:38 2022] POST /identity/v3/projects => generated 369 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:45:38.480943 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab7cb5-a6f3-455c-8d15-f34f3403ef60 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:38.492138 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab7cb5-a6f3-455c-8d15-f34f3403ef60 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:38.555686 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a31d0-aa0a-40b8-82f6-0cdec20004d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:38.556436 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a31d0-aa0a-40b8-82f6-0cdec20004d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:38.556719 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a31d0-aa0a-40b8-82f6-0cdec20004d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:38.557013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a31d0-aa0a-40b8-82f6-0cdec20004d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:38.557691 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a31d0-aa0a-40b8-82f6-0cdec20004d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:38.559631 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a31d0-aa0a-40b8-82f6-0cdec20004d8 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:38.579386 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1306/2616] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:45:38 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:45:38.583338 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d0567-ff60-464f-ac63-a56cce2c4bda admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:38.593868 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d0567-ff60-464f-ac63-a56cce2c4bda admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:38.598459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d8d58-4942-4cdd-97a5-d3a49ba4969a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:38.599107 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d8d58-4942-4cdd-97a5-d3a49ba4969a admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:38.599390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d8d58-4942-4cdd-97a5-d3a49ba4969a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:38.599664 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d8d58-4942-4cdd-97a5-d3a49ba4969a admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:38.600538 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d8d58-4942-4cdd-97a5-d3a49ba4969a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:38.602119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d8d58-4942-4cdd-97a5-d3a49ba4969a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:38.612943 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1311/2617] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:45:38.617702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a31d0-aa0a-40b8-82f6-0cdec20004d8 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:38.630607 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a31d0-aa0a-40b8-82f6-0cdec20004d8 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:38.636083 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2cd94-94dc-4f5c-a808-b22e30b1d38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:38.636973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2cd94-94dc-4f5c-a808-b22e30b1d38b admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:38.637257 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2cd94-94dc-4f5c-a808-b22e30b1d38b admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:38.637535 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2cd94-94dc-4f5c-a808-b22e30b1d38b admin admin] PATH_INFO: `/v3/projects/c161592ffc1842908e2bc050521d7f47/users/c1ee1a6e557a4a36a42dbaaba43ad838/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:38.655516 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2cd94-94dc-4f5c-a808-b22e30b1d38b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c161592ffc1842908e2bc050521d7f47, user_id=c1ee1a6e557a4a36a42dbaaba43ad838, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:38.658258 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2cd94-94dc-4f5c-a808-b22e30b1d38b admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:38.672398 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1307/2618] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:45:38 2022] PUT /identity/v3/projects/c161592ffc1842908e2bc050521d7f47/users/c1ee1a6e557a4a36a42dbaaba43ad838/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:38.676378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d8d58-4942-4cdd-97a5-d3a49ba4969a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:38.687484 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d8d58-4942-4cdd-97a5-d3a49ba4969a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:38.706992 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:38.746324 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da164a7-c03c-4c41-8aa9-d7f0652f5b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:38.746943 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da164a7-c03c-4c41-8aa9-d7f0652f5b1b admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:38.747223 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da164a7-c03c-4c41-8aa9-d7f0652f5b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:38.747495 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da164a7-c03c-4c41-8aa9-d7f0652f5b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:38.748243 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da164a7-c03c-4c41-8aa9-d7f0652f5b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:38.749840 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da164a7-c03c-4c41-8aa9-d7f0652f5b1b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:38.760043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1312/2619] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:45:38.764477 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2cd94-94dc-4f5c-a808-b22e30b1d38b admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:38.769955 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:38.776074 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2cd94-94dc-4f5c-a808-b22e30b1d38b admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:38.780969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae6753-0555-45c4-9de4-bd66b9820528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:38.781847 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae6753-0555-45c4-9de4-bd66b9820528 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:38.782130 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae6753-0555-45c4-9de4-bd66b9820528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:38.782412 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae6753-0555-45c4-9de4-bd66b9820528 admin admin] PATH_INFO: `/v3/projects/c161592ffc1842908e2bc050521d7f47/users/c1ee1a6e557a4a36a42dbaaba43ad838/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:38.784379 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae6753-0555-45c4-9de4-bd66b9820528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c161592ffc1842908e2bc050521d7f47, user_id=c1ee1a6e557a4a36a42dbaaba43ad838, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:38.786985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae6753-0555-45c4-9de4-bd66b9820528 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:38.797655 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1308/2620] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:45:38 2022] PUT /identity/v3/projects/c161592ffc1842908e2bc050521d7f47/users/c1ee1a6e557a4a36a42dbaaba43ad838/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:38.809469 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cbd2c-3518-4237-bc2a-d409769ae9cb None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:38.809895 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cbd2c-3518-4237-bc2a-d409769ae9cb None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:38.810182 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cbd2c-3518-4237-bc2a-d409769ae9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:38.836125 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c81cbd2c-3518-4237-bc2a-d409769ae9cb None None] MFA Rules not processed for user `c1ee1a6e557a4a36a42dbaaba43ad838`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:45:38.837321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c81cbd2c-3518-4237-bc2a-d409769ae9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:38.904380 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1313/2621] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:45:38 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:45:38.916281 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae6753-0555-45c4-9de4-bd66b9820528 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:38.918004 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ceae6753-0555-45c4-9de4-bd66b9820528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:38.976872 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae6753-0555-45c4-9de4-bd66b9820528 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:38.980503 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f7220-543c-4f2b-9103-415a3cf4122e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:38.981137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f7220-543c-4f2b-9103-415a3cf4122e service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:38.981416 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f7220-543c-4f2b-9103-415a3cf4122e service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:38.981688 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f7220-543c-4f2b-9103-415a3cf4122e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:38.990590 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f7220-543c-4f2b-9103-415a3cf4122e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:38.993197 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f7220-543c-4f2b-9103-415a3cf4122e service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:39.002564 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1309/2622] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:45:38 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:45:39.045245 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81cbd2c-3518-4237-bc2a-d409769ae9cb None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:39.046666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c81cbd2c-3518-4237-bc2a-d409769ae9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:39.059931 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81cbd2c-3518-4237-bc2a-d409769ae9cb None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:39.063491 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94649c3-c655-4136-98ec-f894252576a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:39.064160 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94649c3-c655-4136-98ec-f894252576a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:39.064439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94649c3-c655-4136-98ec-f894252576a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:39.064708 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94649c3-c655-4136-98ec-f894252576a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:39.121728 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94649c3-c655-4136-98ec-f894252576a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:39.123155 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94649c3-c655-4136-98ec-f894252576a7 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:39.133000 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1314/2623] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:45:39 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:45:42.380454 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f7220-543c-4f2b-9103-415a3cf4122e service cinder] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:42.443033 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f7220-543c-4f2b-9103-415a3cf4122e service cinder] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:42.447753 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa3698-cd2e-4adb-962d-b9bec50a34a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:42.448639 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa3698-cd2e-4adb-962d-b9bec50a34a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:42.448923 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa3698-cd2e-4adb-962d-b9bec50a34a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:42.449196 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa3698-cd2e-4adb-962d-b9bec50a34a0 admin admin] PATH_INFO: `/v3/users/6d5a63eda5e54075b7f7abeb8297357d` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:42.450362 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aa3698-cd2e-4adb-962d-b9bec50a34a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d5a63eda5e54075b7f7abeb8297357d)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:42.452296 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aa3698-cd2e-4adb-962d-b9bec50a34a0 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:42.496812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-59aa3698-cd2e-4adb-962d-b9bec50a34a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d5a63eda5e54075b7f7abeb8297357d'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:42.503515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-59aa3698-cd2e-4adb-962d-b9bec50a34a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d5a63eda5e54075b7f7abeb8297357d'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:42.510531 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-59aa3698-cd2e-4adb-962d-b9bec50a34a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d5a63eda5e54075b7f7abeb8297357d'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:42.511175 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-59aa3698-cd2e-4adb-962d-b9bec50a34a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d5a63eda5e54075b7f7abeb8297357d'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:42.522930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-59aa3698-cd2e-4adb-962d-b9bec50a34a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d5a63eda5e54075b7f7abeb8297357d'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:42.528709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1310/2624] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:45:42 2022] DELETE /identity/v3/users/6d5a63eda5e54075b7f7abeb8297357d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:42.533129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c7ddb-1e46-4e15-921e-89b2a3824148 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:42.533615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c7ddb-1e46-4e15-921e-89b2a3824148 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:42.533711 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c7ddb-1e46-4e15-921e-89b2a3824148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:42.551044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-274c7ddb-1e46-4e15-921e-89b2a3824148 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:45:42.552396 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-274c7ddb-1e46-4e15-921e-89b2a3824148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:42.592559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:42.613673 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1315/2625] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:45:42.625898 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa3698-cd2e-4adb-962d-b9bec50a34a0 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:42.626873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-59aa3698-cd2e-4adb-962d-b9bec50a34a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:42.642888 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:42.694119 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa3698-cd2e-4adb-962d-b9bec50a34a0 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:42.697812 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb582b42-fea9-48da-afbb-b48669f51c84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:42.698440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb582b42-fea9-48da-afbb-b48669f51c84 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:42.698723 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb582b42-fea9-48da-afbb-b48669f51c84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:42.699050 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb582b42-fea9-48da-afbb-b48669f51c84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:42.708371 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb582b42-fea9-48da-afbb-b48669f51c84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:42.709957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb582b42-fea9-48da-afbb-b48669f51c84 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:42.719523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1311/2626] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:45:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:45:43.176714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c7ddb-1e46-4e15-921e-89b2a3824148 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:43.178365 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-274c7ddb-1e46-4e15-921e-89b2a3824148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:43.191060 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c7ddb-1e46-4e15-921e-89b2a3824148 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:43.196803 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae07cc-ad90-4f11-ade2-af4e89be0824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:43.197809 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae07cc-ad90-4f11-ade2-af4e89be0824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:43.198087 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae07cc-ad90-4f11-ade2-af4e89be0824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:43.198360 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae07cc-ad90-4f11-ade2-af4e89be0824 admin admin] PATH_INFO: `/v3/projects/c0491e6a2f7b46c8bb03afeeb2355e75` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:43.199570 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae07cc-ad90-4f11-ade2-af4e89be0824 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0491e6a2f7b46c8bb03afeeb2355e75)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:43.201833 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae07cc-ad90-4f11-ade2-af4e89be0824 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:43.258001 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3eae07cc-ad90-4f11-ade2-af4e89be0824 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0491e6a2f7b46c8bb03afeeb2355e75', 'request_id': 'req-3eae07cc-ad90-4f11-ade2-af4e89be0824'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:43.266240 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3eae07cc-ad90-4f11-ade2-af4e89be0824 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0491e6a2f7b46c8bb03afeeb2355e75', 'request_id': 'req-3eae07cc-ad90-4f11-ade2-af4e89be0824'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:43.273542 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3eae07cc-ad90-4f11-ade2-af4e89be0824 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0491e6a2f7b46c8bb03afeeb2355e75', 'request_id': 'req-3eae07cc-ad90-4f11-ade2-af4e89be0824'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:43.285454 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3eae07cc-ad90-4f11-ade2-af4e89be0824 admin admin] The token cache is being invalidate because project c0491e6a2f7b46c8bb03afeeb2355e75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:45:43.285790 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3eae07cc-ad90-4f11-ade2-af4e89be0824 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:43.286441 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3eae07cc-ad90-4f11-ade2-af4e89be0824 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:43.288114 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1316/2627] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:45:43 2022] DELETE /identity/v3/projects/c0491e6a2f7b46c8bb03afeeb2355e75 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:43.292040 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb582b42-fea9-48da-afbb-b48669f51c84 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:43.301781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb582b42-fea9-48da-afbb-b48669f51c84 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:43.362665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9166f-acad-40bd-85e4-c38be21cbb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:43.363539 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9166f-acad-40bd-85e4-c38be21cbb21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:43.363848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9166f-acad-40bd-85e4-c38be21cbb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:43.364126 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9166f-acad-40bd-85e4-c38be21cbb21 admin admin] PATH_INFO: `/v3/users/c1ee1a6e557a4a36a42dbaaba43ad838` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:43.366351 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9166f-acad-40bd-85e4-c38be21cbb21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1ee1a6e557a4a36a42dbaaba43ad838)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:43.367887 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9166f-acad-40bd-85e4-c38be21cbb21 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:43.416070 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d9c9166f-acad-40bd-85e4-c38be21cbb21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1ee1a6e557a4a36a42dbaaba43ad838'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:43.423227 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d9c9166f-acad-40bd-85e4-c38be21cbb21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1ee1a6e557a4a36a42dbaaba43ad838'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:43.431235 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d9c9166f-acad-40bd-85e4-c38be21cbb21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1ee1a6e557a4a36a42dbaaba43ad838'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:43.431971 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d9c9166f-acad-40bd-85e4-c38be21cbb21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1ee1a6e557a4a36a42dbaaba43ad838'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:43.444623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d9c9166f-acad-40bd-85e4-c38be21cbb21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1ee1a6e557a4a36a42dbaaba43ad838'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:43.450365 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1312/2628] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:45:43 2022] DELETE /identity/v3/users/c1ee1a6e557a4a36a42dbaaba43ad838 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:43.767613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae07cc-ad90-4f11-ade2-af4e89be0824 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:43.770146 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3eae07cc-ad90-4f11-ade2-af4e89be0824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:43.810094 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae07cc-ad90-4f11-ade2-af4e89be0824 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:43.890871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5368bcb1-4b08-4477-a9cc-ee06fc2de62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:43.891916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368bcb1-4b08-4477-a9cc-ee06fc2de62f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:43.892297 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368bcb1-4b08-4477-a9cc-ee06fc2de62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:43.892677 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368bcb1-4b08-4477-a9cc-ee06fc2de62f admin admin] PATH_INFO: `/v3/projects/c161592ffc1842908e2bc050521d7f47` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:43.894061 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368bcb1-4b08-4477-a9cc-ee06fc2de62f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c161592ffc1842908e2bc050521d7f47)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:43.896123 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368bcb1-4b08-4477-a9cc-ee06fc2de62f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:43.987216 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5368bcb1-4b08-4477-a9cc-ee06fc2de62f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c161592ffc1842908e2bc050521d7f47', 'request_id': 'req-5368bcb1-4b08-4477-a9cc-ee06fc2de62f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:43.997578 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5368bcb1-4b08-4477-a9cc-ee06fc2de62f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c161592ffc1842908e2bc050521d7f47', 'request_id': 'req-5368bcb1-4b08-4477-a9cc-ee06fc2de62f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:44.004661 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5368bcb1-4b08-4477-a9cc-ee06fc2de62f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c161592ffc1842908e2bc050521d7f47', 'request_id': 'req-5368bcb1-4b08-4477-a9cc-ee06fc2de62f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:44.013771 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:44.021593 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5368bcb1-4b08-4477-a9cc-ee06fc2de62f admin admin] The token cache is being invalidate because project c161592ffc1842908e2bc050521d7f47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:45:44.022224 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5368bcb1-4b08-4477-a9cc-ee06fc2de62f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:44.023839 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5368bcb1-4b08-4477-a9cc-ee06fc2de62f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:44.026452 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1317/2629] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:45:43 2022] DELETE /identity/v3/projects/c161592ffc1842908e2bc050521d7f47 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:44.031504 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd54050-07d9-4f36-ac6b-f68d377362dd None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:44.032248 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd54050-07d9-4f36-ac6b-f68d377362dd None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:44.032803 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd54050-07d9-4f36-ac6b-f68d377362dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:44.061014 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-ddd54050-07d9-4f36-ac6b-f68d377362dd None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:45:44.062635 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ddd54050-07d9-4f36-ac6b-f68d377362dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:44.099305 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:44.136464 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1313/2630] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:45:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:45:44.144543 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45026791-99ee-4982-8aa5-90d270ea7799 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:44.144851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45026791-99ee-4982-8aa5-90d270ea7799 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:44.145249 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45026791-99ee-4982-8aa5-90d270ea7799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:44.165373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-45026791-99ee-4982-8aa5-90d270ea7799 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:45:44.166605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-45026791-99ee-4982-8aa5-90d270ea7799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:44.174871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1318/2631] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:45:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:45:44.179909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd54050-07d9-4f36-ac6b-f68d377362dd None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:44.189755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd54050-07d9-4f36-ac6b-f68d377362dd None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:44.194608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85136a1e-f553-47bd-aabd-4391fd3d7b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:44.195259 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85136a1e-f553-47bd-aabd-4391fd3d7b8d admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:44.195552 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85136a1e-f553-47bd-aabd-4391fd3d7b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:44.195847 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85136a1e-f553-47bd-aabd-4391fd3d7b8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:44.196605 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85136a1e-f553-47bd-aabd-4391fd3d7b8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:45:44.196915 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85136a1e-f553-47bd-aabd-4391fd3d7b8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:44.198504 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85136a1e-f553-47bd-aabd-4391fd3d7b8d admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:44.211189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1314/2632] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:45:44.215504 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45026791-99ee-4982-8aa5-90d270ea7799 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:44.226046 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45026791-99ee-4982-8aa5-90d270ea7799 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:44.230981 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf597d68-c88d-49f6-a2e4-c9a48a882964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:44.231668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf597d68-c88d-49f6-a2e4-c9a48a882964 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:44.232000 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf597d68-c88d-49f6-a2e4-c9a48a882964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:44.232300 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf597d68-c88d-49f6-a2e4-c9a48a882964 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:44.233214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf597d68-c88d-49f6-a2e4-c9a48a882964 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:44.235245 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf597d68-c88d-49f6-a2e4-c9a48a882964 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:44.263095 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1319/2633] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:45:44 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:45:44.267369 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85136a1e-f553-47bd-aabd-4391fd3d7b8d admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:44.278218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85136a1e-f553-47bd-aabd-4391fd3d7b8d admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:44.345190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00dfaf-c39f-42ae-b40d-94bef56446e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:44.345872 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00dfaf-c39f-42ae-b40d-94bef56446e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:44.346157 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00dfaf-c39f-42ae-b40d-94bef56446e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:44.346432 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00dfaf-c39f-42ae-b40d-94bef56446e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:44.347566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00dfaf-c39f-42ae-b40d-94bef56446e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:44.349781 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00dfaf-c39f-42ae-b40d-94bef56446e3 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:44.378779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1315/2634] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:45:44 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:45:44.380089 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf597d68-c88d-49f6-a2e4-c9a48a882964 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:44.394221 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf597d68-c88d-49f6-a2e4-c9a48a882964 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:44.400134 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a6f2a-bf62-4caa-a6f7-12a275cf129d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:44.400806 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a6f2a-bf62-4caa-a6f7-12a275cf129d admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:44.401103 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a6f2a-bf62-4caa-a6f7-12a275cf129d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:44.401372 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a6f2a-bf62-4caa-a6f7-12a275cf129d admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:44.402100 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a6f2a-bf62-4caa-a6f7-12a275cf129d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:44.403724 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a6f2a-bf62-4caa-a6f7-12a275cf129d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:44.418576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1320/2635] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:45:44.419832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00dfaf-c39f-42ae-b40d-94bef56446e3 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:44.430581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00dfaf-c39f-42ae-b40d-94bef56446e3 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:44.435430 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7dcd88-2828-4e25-b6fa-d27a6a05f3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:44.436144 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7dcd88-2828-4e25-b6fa-d27a6a05f3c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:44.436428 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7dcd88-2828-4e25-b6fa-d27a6a05f3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:44.436715 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7dcd88-2828-4e25-b6fa-d27a6a05f3c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:44.437439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7dcd88-2828-4e25-b6fa-d27a6a05f3c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:44.439037 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7dcd88-2828-4e25-b6fa-d27a6a05f3c2 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:44.449791 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1316/2636] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:45:44.454118 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a6f2a-bf62-4caa-a6f7-12a275cf129d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:44.465995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a6f2a-bf62-4caa-a6f7-12a275cf129d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:44.471073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ec8ae-c417-4d4a-8525-dba475022430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:44.471652 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ec8ae-c417-4d4a-8525-dba475022430 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:44.471976 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ec8ae-c417-4d4a-8525-dba475022430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:44.472256 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ec8ae-c417-4d4a-8525-dba475022430 admin admin] PATH_INFO: `/v3/projects/8dc8466f21c74dc18ff9429d1d0446d9/users/b2fbc35729ba472984ad3286c1af7bc8/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:44.489679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ec8ae-c417-4d4a-8525-dba475022430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dc8466f21c74dc18ff9429d1d0446d9, user_id=b2fbc35729ba472984ad3286c1af7bc8, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:44.492879 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ec8ae-c417-4d4a-8525-dba475022430 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:44.508994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1321/2637] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:45:44 2022] PUT /identity/v3/projects/8dc8466f21c74dc18ff9429d1d0446d9/users/b2fbc35729ba472984ad3286c1af7bc8/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:45:44.520420 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873fb74-2ae7-4757-b0f6-db9a4d702070 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:44.520718 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873fb74-2ae7-4757-b0f6-db9a4d702070 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:44.521008 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873fb74-2ae7-4757-b0f6-db9a4d702070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:44.553818 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-b873fb74-2ae7-4757-b0f6-db9a4d702070 None None] MFA Rules not processed for user `b2fbc35729ba472984ad3286c1af7bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:45:44.555276 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-b873fb74-2ae7-4757-b0f6-db9a4d702070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:44.639931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1317/2638] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:45:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:45:44.641574 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ec8ae-c417-4d4a-8525-dba475022430 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:44.721495 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ec8ae-c417-4d4a-8525-dba475022430 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:44.727138 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051f6c0-f096-4e0c-8993-ffb239dc6ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:44.727828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051f6c0-f096-4e0c-8993-ffb239dc6ba1 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:44.728259 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051f6c0-f096-4e0c-8993-ffb239dc6ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:44.728539 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051f6c0-f096-4e0c-8993-ffb239dc6ba1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:44.729238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051f6c0-f096-4e0c-8993-ffb239dc6ba1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:44.731150 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051f6c0-f096-4e0c-8993-ffb239dc6ba1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:44.764493 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1322/2639] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:45:44 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:45:44.768602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873fb74-2ae7-4757-b0f6-db9a4d702070 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:44.780788 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873fb74-2ae7-4757-b0f6-db9a4d702070 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:44.812877 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:44.850701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2faff88-fd9e-4333-8113-2f16af822e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:44.851556 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2faff88-fd9e-4333-8113-2f16af822e30 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:44.851889 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2faff88-fd9e-4333-8113-2f16af822e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:44.852167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2faff88-fd9e-4333-8113-2f16af822e30 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:44.852880 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2faff88-fd9e-4333-8113-2f16af822e30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:44.854795 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2faff88-fd9e-4333-8113-2f16af822e30 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:44.895815 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1318/2640] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:45:44 2022] POST /identity/v3/users => generated 296 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:45:44.899963 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051f6c0-f096-4e0c-8993-ffb239dc6ba1 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:44.910934 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051f6c0-f096-4e0c-8993-ffb239dc6ba1 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:44.929777 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7d92f-a272-4269-bce1-e83dd0c516fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:44.930513 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7d92f-a272-4269-bce1-e83dd0c516fc admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:44.930943 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7d92f-a272-4269-bce1-e83dd0c516fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:44.931321 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7d92f-a272-4269-bce1-e83dd0c516fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:44.932323 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7d92f-a272-4269-bce1-e83dd0c516fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:44.934339 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7d92f-a272-4269-bce1-e83dd0c516fc admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:44.950698 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2faff88-fd9e-4333-8113-2f16af822e30 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:44.956390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1323/2641] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:45:44.963292 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2faff88-fd9e-4333-8113-2f16af822e30 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:44.969443 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc404ae-a5dc-45b0-aac9-e8f0e3af3b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:44.970383 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc404ae-a5dc-45b0-aac9-e8f0e3af3b8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:44.970768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc404ae-a5dc-45b0-aac9-e8f0e3af3b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:44.971143 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc404ae-a5dc-45b0-aac9-e8f0e3af3b8a admin admin] PATH_INFO: `/v3/projects/b2e1b9b827df489b848f3aac4c7e04ed/users/88c6d923fdc841a7ae7437e50bfa281b/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:44.997478 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc404ae-a5dc-45b0-aac9-e8f0e3af3b8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2e1b9b827df489b848f3aac4c7e04ed, user_id=88c6d923fdc841a7ae7437e50bfa281b, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:45.001554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc404ae-a5dc-45b0-aac9-e8f0e3af3b8a admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:45.025388 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1319/2642] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:45:44 2022] PUT /identity/v3/projects/b2e1b9b827df489b848f3aac4c7e04ed/users/88c6d923fdc841a7ae7437e50bfa281b/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:45.029145 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7d92f-a272-4269-bce1-e83dd0c516fc admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:45.040073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7d92f-a272-4269-bce1-e83dd0c516fc admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:45.148584 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f33e53-c6f5-46a2-8390-308198a2ada1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:45.149759 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f33e53-c6f5-46a2-8390-308198a2ada1 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:45.150054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f33e53-c6f5-46a2-8390-308198a2ada1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:45.150451 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f33e53-c6f5-46a2-8390-308198a2ada1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:45.151095 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f33e53-c6f5-46a2-8390-308198a2ada1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:45.154086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f33e53-c6f5-46a2-8390-308198a2ada1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:45.180947 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1324/2643] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:45:45.181641 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc404ae-a5dc-45b0-aac9-e8f0e3af3b8a admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:45.192720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc404ae-a5dc-45b0-aac9-e8f0e3af3b8a admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:45.199387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735d487-e56a-44a0-87a1-0220cd19d185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:45.200421 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735d487-e56a-44a0-87a1-0220cd19d185 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:45.200770 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735d487-e56a-44a0-87a1-0220cd19d185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:45.201133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735d487-e56a-44a0-87a1-0220cd19d185 admin admin] PATH_INFO: `/v3/projects/b2e1b9b827df489b848f3aac4c7e04ed/users/88c6d923fdc841a7ae7437e50bfa281b/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:45.203288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5735d487-e56a-44a0-87a1-0220cd19d185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2e1b9b827df489b848f3aac4c7e04ed, user_id=88c6d923fdc841a7ae7437e50bfa281b, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:45.206142 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5735d487-e56a-44a0-87a1-0220cd19d185 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:45.218261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1320/2644] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:45:45 2022] PUT /identity/v3/projects/b2e1b9b827df489b848f3aac4c7e04ed/users/88c6d923fdc841a7ae7437e50bfa281b/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:45:45.228626 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9f133-1b25-4443-b5a0-9f2d4000209e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:45.228946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9f133-1b25-4443-b5a0-9f2d4000209e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:45.229229 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9f133-1b25-4443-b5a0-9f2d4000209e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:45.237493 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:45.266511 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-dcd9f133-1b25-4443-b5a0-9f2d4000209e None None] MFA Rules not processed for user `88c6d923fdc841a7ae7437e50bfa281b`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:45:45.267917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-dcd9f133-1b25-4443-b5a0-9f2d4000209e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:45.341651 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1325/2645] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:45:45 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:45:45.353645 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735d487-e56a-44a0-87a1-0220cd19d185 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:45.355375 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-5735d487-e56a-44a0-87a1-0220cd19d185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:45.418071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735d487-e56a-44a0-87a1-0220cd19d185 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:45.421769 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57adf30-39c4-4e75-b6f7-7f876e9d004b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:45.422439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57adf30-39c4-4e75-b6f7-7f876e9d004b service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:45.422693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57adf30-39c4-4e75-b6f7-7f876e9d004b service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:45.422970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57adf30-39c4-4e75-b6f7-7f876e9d004b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:45.433558 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57adf30-39c4-4e75-b6f7-7f876e9d004b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:45.434989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57adf30-39c4-4e75-b6f7-7f876e9d004b service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:45.444923 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1321/2646] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:45:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:45:45.795639 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9f133-1b25-4443-b5a0-9f2d4000209e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:45.807146 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9f133-1b25-4443-b5a0-9f2d4000209e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:45.814691 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997f166-9967-44af-bcdb-9a3cf2a06aec tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c6d923fdc841a7ae7437e50bfa281b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2e1b9b827df489b848f3aac4c7e04ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:45.815898 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997f166-9967-44af-bcdb-9a3cf2a06aec tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:45.816339 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997f166-9967-44af-bcdb-9a3cf2a06aec tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:45.816725 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997f166-9967-44af-bcdb-9a3cf2a06aec tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:45.819819 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e997f166-9967-44af-bcdb-9a3cf2a06aec tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:45:45.819819 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e997f166-9967-44af-bcdb-9a3cf2a06aec tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:45.821091 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e997f166-9967-44af-bcdb-9a3cf2a06aec tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:45.835383 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1326/2647] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:45:45.839594 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57adf30-39c4-4e75-b6f7-7f876e9d004b service cinder] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:45.851756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57adf30-39c4-4e75-b6f7-7f876e9d004b service cinder] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:45.858133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8021f-257d-492a-aa35-29d946bfbe10 tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c6d923fdc841a7ae7437e50bfa281b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2e1b9b827df489b848f3aac4c7e04ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:45.858935 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8021f-257d-492a-aa35-29d946bfbe10 tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:45.859312 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8021f-257d-492a-aa35-29d946bfbe10 tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:45.859720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8021f-257d-492a-aa35-29d946bfbe10 tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:45.860576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8021f-257d-492a-aa35-29d946bfbe10 tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:45.862697 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8021f-257d-492a-aa35-29d946bfbe10 tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:45.896007 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1322/2648] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:45:45 2022] POST /identity/v3/projects => generated 355 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:45:45.899328 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997f166-9967-44af-bcdb-9a3cf2a06aec tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:45.911998 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997f166-9967-44af-bcdb-9a3cf2a06aec tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:45.973869 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8111f-f71e-4e69-8b94-f0dd0b7f4386 tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c6d923fdc841a7ae7437e50bfa281b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2e1b9b827df489b848f3aac4c7e04ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:45.974459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8111f-f71e-4e69-8b94-f0dd0b7f4386 tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:45.974751 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8111f-f71e-4e69-8b94-f0dd0b7f4386 tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:45.975067 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8111f-f71e-4e69-8b94-f0dd0b7f4386 tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:45.975877 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8111f-f71e-4e69-8b94-f0dd0b7f4386 tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:45:45.976201 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8111f-f71e-4e69-8b94-f0dd0b7f4386 tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:45.977937 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8111f-f71e-4e69-8b94-f0dd0b7f4386 tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:45.991211 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1327/2649] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:45:46.030048 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8021f-257d-492a-aa35-29d946bfbe10 tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:46.041345 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8021f-257d-492a-aa35-29d946bfbe10 tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:46.046055 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a46014-c191-4b2c-813f-02d79790018d tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c6d923fdc841a7ae7437e50bfa281b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b2e1b9b827df489b848f3aac4c7e04ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:46.046910 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a46014-c191-4b2c-813f-02d79790018d tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:46.047198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a46014-c191-4b2c-813f-02d79790018d tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:46.047473 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a46014-c191-4b2c-813f-02d79790018d tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] PATH_INFO: `/v3/projects/05593298a9bb48a5acd750505b35c7fc` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:46.048694 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a46014-c191-4b2c-813f-02d79790018d tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] RBAC: Authorizing `identity:delete_project(project_id=05593298a9bb48a5acd750505b35c7fc)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:46.050585 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a46014-c191-4b2c-813f-02d79790018d tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:46.099553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-49a46014-c191-4b2c-813f-02d79790018d tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05593298a9bb48a5acd750505b35c7fc', 'request_id': 'req-49a46014-c191-4b2c-813f-02d79790018d'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:46.104205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=88,keystone:INSERT=6,keystone:DELETE=6 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:46.105888 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-49a46014-c191-4b2c-813f-02d79790018d tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05593298a9bb48a5acd750505b35c7fc', 'request_id': 'req-49a46014-c191-4b2c-813f-02d79790018d'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:46.113767 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-49a46014-c191-4b2c-813f-02d79790018d tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05593298a9bb48a5acd750505b35c7fc', 'request_id': 'req-49a46014-c191-4b2c-813f-02d79790018d'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:46.126150 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-49a46014-c191-4b2c-813f-02d79790018d tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] The token cache is being invalidate because project 05593298a9bb48a5acd750505b35c7fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:45:46.126441 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-49a46014-c191-4b2c-813f-02d79790018d tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:46.127120 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-49a46014-c191-4b2c-813f-02d79790018d tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:46.128713 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1323/2650] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:45:46 2022] DELETE /identity/v3/projects/05593298a9bb48a5acd750505b35c7fc => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:46.416737 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8111f-f71e-4e69-8b94-f0dd0b7f4386 tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:46.419111 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4eb8111f-f71e-4e69-8b94-f0dd0b7f4386 tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:46.499102 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8111f-f71e-4e69-8b94-f0dd0b7f4386 tempest-VolumeQuotaClassesTest-469724133 tempest-VolumeQuotaClassesTest-469724133-project-admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:46.512263 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:46.577351 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4e466-dc09-47e7-b7ca-fbc8ac2a0b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:46.578224 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4e466-dc09-47e7-b7ca-fbc8ac2a0b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:46.578505 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4e466-dc09-47e7-b7ca-fbc8ac2a0b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:46.578774 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4e466-dc09-47e7-b7ca-fbc8ac2a0b3e admin admin] PATH_INFO: `/v3/users/b2fbc35729ba472984ad3286c1af7bc8` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:46.579989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4e466-dc09-47e7-b7ca-fbc8ac2a0b3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2fbc35729ba472984ad3286c1af7bc8)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:46.581896 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4e466-dc09-47e7-b7ca-fbc8ac2a0b3e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:46.635722 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-83e4e466-dc09-47e7-b7ca-fbc8ac2a0b3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2fbc35729ba472984ad3286c1af7bc8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:46.643097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-83e4e466-dc09-47e7-b7ca-fbc8ac2a0b3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2fbc35729ba472984ad3286c1af7bc8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:46.643777 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-83e4e466-dc09-47e7-b7ca-fbc8ac2a0b3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2fbc35729ba472984ad3286c1af7bc8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:46.644498 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-83e4e466-dc09-47e7-b7ca-fbc8ac2a0b3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2fbc35729ba472984ad3286c1af7bc8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:46.657834 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-83e4e466-dc09-47e7-b7ca-fbc8ac2a0b3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2fbc35729ba472984ad3286c1af7bc8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:46.680958 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1328/2651] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:45:46 2022] DELETE /identity/v3/users/b2fbc35729ba472984ad3286c1af7bc8 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:46.685766 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fa3d2-78c7-4a33-8cad-0293566d4e19 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:46.686363 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fa3d2-78c7-4a33-8cad-0293566d4e19 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:46.686363 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fa3d2-78c7-4a33-8cad-0293566d4e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:46.707070 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-c29fa3d2-78c7-4a33-8cad-0293566d4e19 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:45:46.708312 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-c29fa3d2-78c7-4a33-8cad-0293566d4e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:46.744742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c8984-0a4a-496b-8982-ebca8210acc9 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:46.744967 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c8984-0a4a-496b-8982-ebca8210acc9 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:46.745253 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c8984-0a4a-496b-8982-ebca8210acc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:46.762311 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3d2c8984-0a4a-496b-8982-ebca8210acc9 None None] MFA Rules not processed for user `7ba341ea1ac34874bbfb8b2f713aea1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:45:46.763807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3d2c8984-0a4a-496b-8982-ebca8210acc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:46.782928 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1324/2652] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:45:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:45:46.794599 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fa3d2-78c7-4a33-8cad-0293566d4e19 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:46.795958 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-c29fa3d2-78c7-4a33-8cad-0293566d4e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:46.821464 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1329/2653] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:45:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:45:46.858471 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fa3d2-78c7-4a33-8cad-0293566d4e19 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:46.862055 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5087b-a55f-406e-966e-aa5f44180cc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:46.862701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5087b-a55f-406e-966e-aa5f44180cc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:46.862985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5087b-a55f-406e-966e-aa5f44180cc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:46.863261 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5087b-a55f-406e-966e-aa5f44180cc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:46.873665 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b5087b-a55f-406e-966e-aa5f44180cc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:46.874516 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b5087b-a55f-406e-966e-aa5f44180cc6 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:46.884895 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1325/2654] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:45:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:45:47.304150 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c8984-0a4a-496b-8982-ebca8210acc9 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:47.305948 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3d2c8984-0a4a-496b-8982-ebca8210acc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:47.327767 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c8984-0a4a-496b-8982-ebca8210acc9 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:47.332728 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11a232-c3c9-447f-9466-08f00b2b5092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:47.333752 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11a232-c3c9-447f-9466-08f00b2b5092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:47.334673 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11a232-c3c9-447f-9466-08f00b2b5092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:47.335054 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11a232-c3c9-447f-9466-08f00b2b5092 admin admin] PATH_INFO: `/v3/projects/8dc8466f21c74dc18ff9429d1d0446d9` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:47.336417 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df11a232-c3c9-447f-9466-08f00b2b5092 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dc8466f21c74dc18ff9429d1d0446d9)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:47.338342 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df11a232-c3c9-447f-9466-08f00b2b5092 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:47.385225 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:47.397130 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-df11a232-c3c9-447f-9466-08f00b2b5092 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dc8466f21c74dc18ff9429d1d0446d9', 'request_id': 'req-df11a232-c3c9-447f-9466-08f00b2b5092'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:47.405804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-df11a232-c3c9-447f-9466-08f00b2b5092 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dc8466f21c74dc18ff9429d1d0446d9', 'request_id': 'req-df11a232-c3c9-447f-9466-08f00b2b5092'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:47.411545 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-df11a232-c3c9-447f-9466-08f00b2b5092 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dc8466f21c74dc18ff9429d1d0446d9', 'request_id': 'req-df11a232-c3c9-447f-9466-08f00b2b5092'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:47.423303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-df11a232-c3c9-447f-9466-08f00b2b5092 admin admin] The token cache is being invalidate because project 8dc8466f21c74dc18ff9429d1d0446d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:45:47.423673 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-df11a232-c3c9-447f-9466-08f00b2b5092 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:47.424375 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-df11a232-c3c9-447f-9466-08f00b2b5092 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:47.425958 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1330/2655] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:45:47 2022] DELETE /identity/v3/projects/8dc8466f21c74dc18ff9429d1d0446d9 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:47.430681 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5087b-a55f-406e-966e-aa5f44180cc6 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:47.440384 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5087b-a55f-406e-966e-aa5f44180cc6 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:47.499519 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad81889-cd54-4533-97d3-0a33f9b81bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:47.500429 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad81889-cd54-4533-97d3-0a33f9b81bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:47.500710 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad81889-cd54-4533-97d3-0a33f9b81bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:47.500982 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad81889-cd54-4533-97d3-0a33f9b81bad admin admin] PATH_INFO: `/v3/users/88c6d923fdc841a7ae7437e50bfa281b` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:47.502151 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad81889-cd54-4533-97d3-0a33f9b81bad admin admin] RBAC: Authorizing `identity:delete_user(user_id=88c6d923fdc841a7ae7437e50bfa281b)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:47.504110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad81889-cd54-4533-97d3-0a33f9b81bad admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:47.545228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:47.564640 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-dad81889-cd54-4533-97d3-0a33f9b81bad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88c6d923fdc841a7ae7437e50bfa281b'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:47.573891 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-dad81889-cd54-4533-97d3-0a33f9b81bad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88c6d923fdc841a7ae7437e50bfa281b'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:47.583829 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-dad81889-cd54-4533-97d3-0a33f9b81bad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88c6d923fdc841a7ae7437e50bfa281b'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:47.586008 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-dad81889-cd54-4533-97d3-0a33f9b81bad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88c6d923fdc841a7ae7437e50bfa281b'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:47.602669 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-dad81889-cd54-4533-97d3-0a33f9b81bad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88c6d923fdc841a7ae7437e50bfa281b'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:47.612677 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1326/2656] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:45:47 2022] DELETE /identity/v3/users/88c6d923fdc841a7ae7437e50bfa281b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:47.992162 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11a232-c3c9-447f-9466-08f00b2b5092 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:47.993743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-df11a232-c3c9-447f-9466-08f00b2b5092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:48.007955 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11a232-c3c9-447f-9466-08f00b2b5092 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:48.080558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059aab35-c870-4471-a1df-b9022259d33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:48.081427 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059aab35-c870-4471-a1df-b9022259d33e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:48.081726 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059aab35-c870-4471-a1df-b9022259d33e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:48.081983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059aab35-c870-4471-a1df-b9022259d33e admin admin] PATH_INFO: `/v3/projects/b2e1b9b827df489b848f3aac4c7e04ed` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:48.083158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059aab35-c870-4471-a1df-b9022259d33e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2e1b9b827df489b848f3aac4c7e04ed)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:48.085713 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059aab35-c870-4471-a1df-b9022259d33e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:48.151052 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-059aab35-c870-4471-a1df-b9022259d33e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2e1b9b827df489b848f3aac4c7e04ed', 'request_id': 'req-059aab35-c870-4471-a1df-b9022259d33e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:48.159082 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-059aab35-c870-4471-a1df-b9022259d33e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2e1b9b827df489b848f3aac4c7e04ed', 'request_id': 'req-059aab35-c870-4471-a1df-b9022259d33e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:48.167631 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-059aab35-c870-4471-a1df-b9022259d33e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2e1b9b827df489b848f3aac4c7e04ed', 'request_id': 'req-059aab35-c870-4471-a1df-b9022259d33e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:48.183584 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-059aab35-c870-4471-a1df-b9022259d33e admin admin] The token cache is being invalidate because project b2e1b9b827df489b848f3aac4c7e04ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:45:48.183978 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-059aab35-c870-4471-a1df-b9022259d33e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:48.184773 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-059aab35-c870-4471-a1df-b9022259d33e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:45:48.186761 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1331/2657] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:45:47 2022] DELETE /identity/v3/projects/b2e1b9b827df489b848f3aac4c7e04ed => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:48.192633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed0ff6-4fe5-414b-8568-d45eb39ccc45 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:48.193212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed0ff6-4fe5-414b-8568-d45eb39ccc45 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:48.193212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed0ff6-4fe5-414b-8568-d45eb39ccc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:48.211557 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-67ed0ff6-4fe5-414b-8568-d45eb39ccc45 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:45:48.212794 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-67ed0ff6-4fe5-414b-8568-d45eb39ccc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:48.274417 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1327/2658] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:45:48.282199 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38e19a-acc8-4574-8521-6fb504ed3e75 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:48.282504 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38e19a-acc8-4574-8521-6fb504ed3e75 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:48.282784 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38e19a-acc8-4574-8521-6fb504ed3e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:48.302381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-bf38e19a-acc8-4574-8521-6fb504ed3e75 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:45:48.304385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bf38e19a-acc8-4574-8521-6fb504ed3e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:48.314567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1332/2659] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:45:48.321205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed0ff6-4fe5-414b-8568-d45eb39ccc45 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:48.331560 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed0ff6-4fe5-414b-8568-d45eb39ccc45 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:48.336549 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875487e-af26-432e-a95d-436235122193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:48.337212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875487e-af26-432e-a95d-436235122193 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:48.337493 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875487e-af26-432e-a95d-436235122193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:48.337769 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875487e-af26-432e-a95d-436235122193 admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:48.338494 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875487e-af26-432e-a95d-436235122193 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:45:48.338800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875487e-af26-432e-a95d-436235122193 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:48.340720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875487e-af26-432e-a95d-436235122193 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:48.356105 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1328/2660] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:45:48.358147 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38e19a-acc8-4574-8521-6fb504ed3e75 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:48.368477 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38e19a-acc8-4574-8521-6fb504ed3e75 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:48.373662 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34581b2-e360-4557-be75-a92fd9781791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:48.374352 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34581b2-e360-4557-be75-a92fd9781791 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:48.374635 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34581b2-e360-4557-be75-a92fd9781791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:48.374958 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34581b2-e360-4557-be75-a92fd9781791 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:48.375650 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34581b2-e360-4557-be75-a92fd9781791 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:48.377610 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34581b2-e360-4557-be75-a92fd9781791 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:48.440559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1333/2661] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:45:48 2022] POST /identity/v3/projects => generated 375 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:45:48.445517 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875487e-af26-432e-a95d-436235122193 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:48.461349 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875487e-af26-432e-a95d-436235122193 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:48.531609 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4fca5-0f96-4942-bb41-dbea20f19c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:48.532514 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4fca5-0f96-4942-bb41-dbea20f19c18 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:48.532801 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4fca5-0f96-4942-bb41-dbea20f19c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:48.533347 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4fca5-0f96-4942-bb41-dbea20f19c18 admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:48.533771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4fca5-0f96-4942-bb41-dbea20f19c18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:48.535882 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4fca5-0f96-4942-bb41-dbea20f19c18 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:48.560184 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1329/2662] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:45:48 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:45:48.564549 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34581b2-e360-4557-be75-a92fd9781791 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:48.576942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34581b2-e360-4557-be75-a92fd9781791 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:48.581490 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a447d-54c5-451c-9bd8-102ac222f931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:48.582183 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a447d-54c5-451c-9bd8-102ac222f931 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:48.582465 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a447d-54c5-451c-9bd8-102ac222f931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:48.582742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a447d-54c5-451c-9bd8-102ac222f931 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:48.583506 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a447d-54c5-451c-9bd8-102ac222f931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:48.585359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a447d-54c5-451c-9bd8-102ac222f931 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:48.596817 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1334/2663] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:45:48.601008 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4fca5-0f96-4942-bb41-dbea20f19c18 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:48.610867 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4fca5-0f96-4942-bb41-dbea20f19c18 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:48.616202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7150d-fe97-41b8-882e-e902d039d3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:48.616819 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7150d-fe97-41b8-882e-e902d039d3e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:48.617100 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7150d-fe97-41b8-882e-e902d039d3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:48.617370 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7150d-fe97-41b8-882e-e902d039d3e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:48.618092 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7150d-fe97-41b8-882e-e902d039d3e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:48.619672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7150d-fe97-41b8-882e-e902d039d3e2 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:48.629610 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1330/2664] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:45:48.634326 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a447d-54c5-451c-9bd8-102ac222f931 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:48.644339 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a447d-54c5-451c-9bd8-102ac222f931 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:48.649164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59699309-b97d-4036-a806-e739d43d0555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:48.650030 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59699309-b97d-4036-a806-e739d43d0555 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:48.650312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59699309-b97d-4036-a806-e739d43d0555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:48.650587 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59699309-b97d-4036-a806-e739d43d0555 admin admin] PATH_INFO: `/v3/projects/772bb667f78d4f13a13f2444249101d0/users/98ea4d3b3fca457db3298c0a28f26a7a/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:48.662074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:48.666084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59699309-b97d-4036-a806-e739d43d0555 admin admin] RBAC: Authorizing `identity:create_grant(project_id=772bb667f78d4f13a13f2444249101d0, user_id=98ea4d3b3fca457db3298c0a28f26a7a, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:48.669393 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59699309-b97d-4036-a806-e739d43d0555 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:48.684603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1335/2665] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:45:48 2022] PUT /identity/v3/projects/772bb667f78d4f13a13f2444249101d0/users/98ea4d3b3fca457db3298c0a28f26a7a/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:45:48.696724 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b2026-1010-4dd9-a5ab-336159bb65b8 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:48.697018 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b2026-1010-4dd9-a5ab-336159bb65b8 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:48.697347 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b2026-1010-4dd9-a5ab-336159bb65b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:48.721307 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:48.725514 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-9e6b2026-1010-4dd9-a5ab-336159bb65b8 None None] MFA Rules not processed for user `98ea4d3b3fca457db3298c0a28f26a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:45:48.727649 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-9e6b2026-1010-4dd9-a5ab-336159bb65b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:48.809423 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1331/2666] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:45:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:45:48.822181 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59699309-b97d-4036-a806-e739d43d0555 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:45:48.824402 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-59699309-b97d-4036-a806-e739d43d0555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:48.918873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59699309-b97d-4036-a806-e739d43d0555 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:45:48.925293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41c75c-1d48-4119-a37a-a59de816669f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:45:48.926124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41c75c-1d48-4119-a37a-a59de816669f service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:48.926881 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41c75c-1d48-4119-a37a-a59de816669f service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:48.927318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41c75c-1d48-4119-a37a-a59de816669f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:48.939575 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e41c75c-1d48-4119-a37a-a59de816669f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:45:48.942154 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e41c75c-1d48-4119-a37a-a59de816669f service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:45:48.953711 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1336/2667] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:45:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:45:58.806253 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=26 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:58.949293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=29,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:45:59.892897 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d608f-c611-4cbe-a34e-66ecb13cc349 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:45:59.893405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d608f-c611-4cbe-a34e-66ecb13cc349 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:45:59.893504 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d608f-c611-4cbe-a34e-66ecb13cc349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:45:59.911273 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-1b4d608f-c611-4cbe-a34e-66ecb13cc349 None None] MFA Rules not processed for user `63d02effcaa64a2580cdd1613e7ff1d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:45:59.913059 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-1b4d608f-c611-4cbe-a34e-66ecb13cc349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:45:59.966903 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1332/2668] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:45:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:46:09.960236 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=25 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:46:10.330863 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41c75c-1d48-4119-a37a-a59de816669f service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:10.334138 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3e41c75c-1d48-4119-a37a-a59de816669f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:10.443601 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41c75c-1d48-4119-a37a-a59de816669f service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:10.452174 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e697dc-b28e-4cd4-86cc-ca679fffa539 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361be69f1c0a41c18afd206bccf56447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:10.452926 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e697dc-b28e-4cd4-86cc-ca679fffa539 service placement] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:10.453282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e697dc-b28e-4cd4-86cc-ca679fffa539 service placement] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:10.453629 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e697dc-b28e-4cd4-86cc-ca679fffa539 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:10.455674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-81e697dc-b28e-4cd4-86cc-ca679fffa539 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:10.472546 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e697dc-b28e-4cd4-86cc-ca679fffa539 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:10.474376 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e697dc-b28e-4cd4-86cc-ca679fffa539 service placement] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:10.497282 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1337/2669] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:46:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:46:13.018996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d608f-c611-4cbe-a34e-66ecb13cc349 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:13.104720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d608f-c611-4cbe-a34e-66ecb13cc349 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:13.110345 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99dca2-81ba-40e6-a280-641d19bb386f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:13.111273 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99dca2-81ba-40e6-a280-641d19bb386f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:13.111556 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99dca2-81ba-40e6-a280-641d19bb386f admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:13.111857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99dca2-81ba-40e6-a280-641d19bb386f admin admin] PATH_INFO: `/v3/users/98ea4d3b3fca457db3298c0a28f26a7a` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:13.114093 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99dca2-81ba-40e6-a280-641d19bb386f admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ea4d3b3fca457db3298c0a28f26a7a)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:13.117019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99dca2-81ba-40e6-a280-641d19bb386f admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:13.179745 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6a99dca2-81ba-40e6-a280-641d19bb386f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ea4d3b3fca457db3298c0a28f26a7a'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:13.186825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6a99dca2-81ba-40e6-a280-641d19bb386f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ea4d3b3fca457db3298c0a28f26a7a'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:13.195946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6a99dca2-81ba-40e6-a280-641d19bb386f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ea4d3b3fca457db3298c0a28f26a7a'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:13.196576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6a99dca2-81ba-40e6-a280-641d19bb386f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ea4d3b3fca457db3298c0a28f26a7a'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:13.210854 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-6a99dca2-81ba-40e6-a280-641d19bb386f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ea4d3b3fca457db3298c0a28f26a7a'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:13.217282 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1333/2670] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:46:13 2022] DELETE /identity/v3/users/98ea4d3b3fca457db3298c0a28f26a7a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:46:13.221574 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8a69a-bb40-4c57-a7cb-aa0932c03567 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:13.222060 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8a69a-bb40-4c57-a7cb-aa0932c03567 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:13.222176 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8a69a-bb40-4c57-a7cb-aa0932c03567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:13.239486 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-5de8a69a-bb40-4c57-a7cb-aa0932c03567 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:46:13.240845 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5de8a69a-bb40-4c57-a7cb-aa0932c03567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:13.306203 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1338/2671] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:46:13.315936 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99dca2-81ba-40e6-a280-641d19bb386f admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:13.318444 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-6a99dca2-81ba-40e6-a280-641d19bb386f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:13.376623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99dca2-81ba-40e6-a280-641d19bb386f admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:13.381732 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14288e06-63ba-48bc-92f0-0181116c4020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:13.382390 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14288e06-63ba-48bc-92f0-0181116c4020 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:13.382682 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14288e06-63ba-48bc-92f0-0181116c4020 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:13.382956 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14288e06-63ba-48bc-92f0-0181116c4020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:13.392824 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14288e06-63ba-48bc-92f0-0181116c4020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:13.394247 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14288e06-63ba-48bc-92f0-0181116c4020 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:13.404396 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1334/2672] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:46:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:46:13.677884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8a69a-bb40-4c57-a7cb-aa0932c03567 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:13.679565 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5de8a69a-bb40-4c57-a7cb-aa0932c03567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:13.693740 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8a69a-bb40-4c57-a7cb-aa0932c03567 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:13.699835 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b107cf-97cc-4d95-b14b-1539568bd1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:13.700528 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b107cf-97cc-4d95-b14b-1539568bd1f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:13.700872 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b107cf-97cc-4d95-b14b-1539568bd1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:13.701217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b107cf-97cc-4d95-b14b-1539568bd1f8 admin admin] PATH_INFO: `/v3/projects/772bb667f78d4f13a13f2444249101d0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:13.702681 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b107cf-97cc-4d95-b14b-1539568bd1f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=772bb667f78d4f13a13f2444249101d0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:13.706382 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b107cf-97cc-4d95-b14b-1539568bd1f8 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:13.758289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f4b107cf-97cc-4d95-b14b-1539568bd1f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '772bb667f78d4f13a13f2444249101d0', 'request_id': 'req-f4b107cf-97cc-4d95-b14b-1539568bd1f8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:13.766467 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f4b107cf-97cc-4d95-b14b-1539568bd1f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '772bb667f78d4f13a13f2444249101d0', 'request_id': 'req-f4b107cf-97cc-4d95-b14b-1539568bd1f8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:13.772770 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f4b107cf-97cc-4d95-b14b-1539568bd1f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '772bb667f78d4f13a13f2444249101d0', 'request_id': 'req-f4b107cf-97cc-4d95-b14b-1539568bd1f8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:13.784048 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f4b107cf-97cc-4d95-b14b-1539568bd1f8 admin admin] The token cache is being invalidate because project 772bb667f78d4f13a13f2444249101d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:46:13.784340 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f4b107cf-97cc-4d95-b14b-1539568bd1f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:13.784990 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f4b107cf-97cc-4d95-b14b-1539568bd1f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:13.786644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1339/2673] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:46:13 2022] DELETE /identity/v3/projects/772bb667f78d4f13a13f2444249101d0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:46:13.791704 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6f514-b480-4a1d-ae9a-8a73bdab7f59 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:13.792102 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6f514-b480-4a1d-ae9a-8a73bdab7f59 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:13.792471 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6f514-b480-4a1d-ae9a-8a73bdab7f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:13.812290 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-87e6f514-b480-4a1d-ae9a-8a73bdab7f59 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:46:13.813487 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-87e6f514-b480-4a1d-ae9a-8a73bdab7f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:13.831809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:46:13.877203 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1335/2674] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:46:13.885550 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c35d1-6d95-44e3-bf94-2f06170d0200 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:13.885844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c35d1-6d95-44e3-bf94-2f06170d0200 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:13.886126 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c35d1-6d95-44e3-bf94-2f06170d0200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:13.904958 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-330c35d1-6d95-44e3-bf94-2f06170d0200 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:46:13.906177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-330c35d1-6d95-44e3-bf94-2f06170d0200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:13.914773 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1340/2675] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:46:13.919327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6f514-b480-4a1d-ae9a-8a73bdab7f59 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:13.931130 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6f514-b480-4a1d-ae9a-8a73bdab7f59 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:13.935671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92df19-0931-4852-8966-337aa3267cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:13.936581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92df19-0931-4852-8966-337aa3267cde admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:13.936867 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92df19-0931-4852-8966-337aa3267cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:13.937141 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92df19-0931-4852-8966-337aa3267cde admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:13.937949 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92df19-0931-4852-8966-337aa3267cde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:46:13.938255 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92df19-0931-4852-8966-337aa3267cde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:13.939854 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92df19-0931-4852-8966-337aa3267cde admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:13.953782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1336/2676] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:46:13.958375 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330c35d1-6d95-44e3-bf94-2f06170d0200 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:13.967912 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330c35d1-6d95-44e3-bf94-2f06170d0200 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:13.972714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416abb9a-a6cc-4605-8d6f-f2b2886d4c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:13.973900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416abb9a-a6cc-4605-8d6f-f2b2886d4c8a admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:13.974183 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416abb9a-a6cc-4605-8d6f-f2b2886d4c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:13.974459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416abb9a-a6cc-4605-8d6f-f2b2886d4c8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:13.975156 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416abb9a-a6cc-4605-8d6f-f2b2886d4c8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:13.977075 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416abb9a-a6cc-4605-8d6f-f2b2886d4c8a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:13.980488 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:46:14.006498 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1341/2677] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:46:13 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:46:14.010709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92df19-0931-4852-8966-337aa3267cde admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:14.022948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92df19-0931-4852-8966-337aa3267cde admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:14.097192 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addbcb4-3bb3-4eb0-808c-376449703f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:14.098037 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addbcb4-3bb3-4eb0-808c-376449703f0a admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:14.099065 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addbcb4-3bb3-4eb0-808c-376449703f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:14.099441 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addbcb4-3bb3-4eb0-808c-376449703f0a admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:14.100355 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4addbcb4-3bb3-4eb0-808c-376449703f0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:14.102444 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4addbcb4-3bb3-4eb0-808c-376449703f0a admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:14.123464 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1337/2678] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:46:14 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:46:14.127449 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416abb9a-a6cc-4605-8d6f-f2b2886d4c8a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:14.138185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416abb9a-a6cc-4605-8d6f-f2b2886d4c8a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:14.142969 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8da757-ed93-45c1-a24d-62e7474da2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:14.143646 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8da757-ed93-45c1-a24d-62e7474da2c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:14.143955 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8da757-ed93-45c1-a24d-62e7474da2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:14.144220 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8da757-ed93-45c1-a24d-62e7474da2c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:14.144937 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8da757-ed93-45c1-a24d-62e7474da2c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:14.146544 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8da757-ed93-45c1-a24d-62e7474da2c4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:14.156623 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1342/2679] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:46:14.161211 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addbcb4-3bb3-4eb0-808c-376449703f0a admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:14.172121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addbcb4-3bb3-4eb0-808c-376449703f0a admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:14.177454 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1630b6-bf70-43b9-a863-2e5ecfa7da63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:14.178162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1630b6-bf70-43b9-a863-2e5ecfa7da63 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:14.178462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1630b6-bf70-43b9-a863-2e5ecfa7da63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:14.178718 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1630b6-bf70-43b9-a863-2e5ecfa7da63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:14.179441 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1630b6-bf70-43b9-a863-2e5ecfa7da63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:14.181309 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1630b6-bf70-43b9-a863-2e5ecfa7da63 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:14.194451 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1338/2680] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:46:14.196095 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8da757-ed93-45c1-a24d-62e7474da2c4 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:14.206068 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8da757-ed93-45c1-a24d-62e7474da2c4 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:14.211433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4ba61-a4c3-4ee9-b5fa-3a5c24464e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:14.212412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4ba61-a4c3-4ee9-b5fa-3a5c24464e81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:14.212697 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4ba61-a4c3-4ee9-b5fa-3a5c24464e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:14.212972 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4ba61-a4c3-4ee9-b5fa-3a5c24464e81 admin admin] PATH_INFO: `/v3/projects/6dbd749e7fce4a19839deccdad621f68/users/42de6891265345f7b60ca34d6c88cdc3/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:14.228405 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc4ba61-a4c3-4ee9-b5fa-3a5c24464e81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dbd749e7fce4a19839deccdad621f68, user_id=42de6891265345f7b60ca34d6c88cdc3, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:14.231640 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc4ba61-a4c3-4ee9-b5fa-3a5c24464e81 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:14.244903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1343/2681] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:46:14 2022] PUT /identity/v3/projects/6dbd749e7fce4a19839deccdad621f68/users/42de6891265345f7b60ca34d6c88cdc3/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:46:14.256896 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc98ba-e9ff-44aa-a003-1968193978f5 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:14.257195 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc98ba-e9ff-44aa-a003-1968193978f5 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:14.257476 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc98ba-e9ff-44aa-a003-1968193978f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:14.286623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-eedc98ba-e9ff-44aa-a003-1968193978f5 None None] MFA Rules not processed for user `42de6891265345f7b60ca34d6c88cdc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:46:14.287964 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-eedc98ba-e9ff-44aa-a003-1968193978f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:14.356387 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1339/2682] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:46:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:46:14.368097 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4ba61-a4c3-4ee9-b5fa-3a5c24464e81 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:14.369946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5fc4ba61-a4c3-4ee9-b5fa-3a5c24464e81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:14.428230 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4ba61-a4c3-4ee9-b5fa-3a5c24464e81 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:14.431794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64386b-8942-441b-b418-2da1e308edc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:14.432449 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64386b-8942-441b-b418-2da1e308edc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:14.432736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64386b-8942-441b-b418-2da1e308edc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:14.433007 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64386b-8942-441b-b418-2da1e308edc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:14.441228 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64386b-8942-441b-b418-2da1e308edc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:14.442658 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64386b-8942-441b-b418-2da1e308edc4 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:14.451375 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1344/2683] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:46:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:46:14.475659 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc98ba-e9ff-44aa-a003-1968193978f5 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:14.494920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:46:14.539680 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc98ba-e9ff-44aa-a003-1968193978f5 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:14.544372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee81e7-4b6d-4d13-b7b7-46d3813375d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:14.545249 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee81e7-4b6d-4d13-b7b7-46d3813375d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:14.545530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee81e7-4b6d-4d13-b7b7-46d3813375d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:14.545804 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee81e7-4b6d-4d13-b7b7-46d3813375d7 admin admin] PATH_INFO: `/v3/users/42de6891265345f7b60ca34d6c88cdc3` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:14.546989 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee81e7-4b6d-4d13-b7b7-46d3813375d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42de6891265345f7b60ca34d6c88cdc3)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:14.549000 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee81e7-4b6d-4d13-b7b7-46d3813375d7 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:14.601566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-80ee81e7-4b6d-4d13-b7b7-46d3813375d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42de6891265345f7b60ca34d6c88cdc3'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:14.608593 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-80ee81e7-4b6d-4d13-b7b7-46d3813375d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42de6891265345f7b60ca34d6c88cdc3'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:14.616906 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-80ee81e7-4b6d-4d13-b7b7-46d3813375d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42de6891265345f7b60ca34d6c88cdc3'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:14.617586 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-80ee81e7-4b6d-4d13-b7b7-46d3813375d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42de6891265345f7b60ca34d6c88cdc3'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:14.629995 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-80ee81e7-4b6d-4d13-b7b7-46d3813375d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42de6891265345f7b60ca34d6c88cdc3'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:14.635954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1340/2684] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:46:14 2022] DELETE /identity/v3/users/42de6891265345f7b60ca34d6c88cdc3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:46:14.640739 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40beb2a-b2a8-4ef9-b889-67eeccbd15ed None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:14.641104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40beb2a-b2a8-4ef9-b889-67eeccbd15ed None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:14.641488 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40beb2a-b2a8-4ef9-b889-67eeccbd15ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:14.658992 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-e40beb2a-b2a8-4ef9-b889-67eeccbd15ed None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:46:14.660281 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e40beb2a-b2a8-4ef9-b889-67eeccbd15ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:14.718035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1345/2685] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:46:14.727671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee81e7-4b6d-4d13-b7b7-46d3813375d7 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:14.729127 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-80ee81e7-4b6d-4d13-b7b7-46d3813375d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:14.786224 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee81e7-4b6d-4d13-b7b7-46d3813375d7 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:14.790053 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ffa284-dc8e-40da-8cb9-c699ff155c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:14.790693 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffa284-dc8e-40da-8cb9-c699ff155c14 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:14.790978 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffa284-dc8e-40da-8cb9-c699ff155c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:14.791254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffa284-dc8e-40da-8cb9-c699ff155c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:14.800518 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ffa284-dc8e-40da-8cb9-c699ff155c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:14.801982 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ffa284-dc8e-40da-8cb9-c699ff155c14 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:14.812833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1341/2686] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:46:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:46:15.091917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40beb2a-b2a8-4ef9-b889-67eeccbd15ed None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:15.093390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e40beb2a-b2a8-4ef9-b889-67eeccbd15ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:15.114446 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40beb2a-b2a8-4ef9-b889-67eeccbd15ed None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:15.120810 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cc8b69-ec00-45dc-8777-f4a6400a3284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:15.121802 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc8b69-ec00-45dc-8777-f4a6400a3284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:15.122174 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc8b69-ec00-45dc-8777-f4a6400a3284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:15.122327 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc8b69-ec00-45dc-8777-f4a6400a3284 admin admin] PATH_INFO: `/v3/projects/6dbd749e7fce4a19839deccdad621f68` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:15.123524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cc8b69-ec00-45dc-8777-f4a6400a3284 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dbd749e7fce4a19839deccdad621f68)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:15.125501 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cc8b69-ec00-45dc-8777-f4a6400a3284 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:15.173049 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:46:15.174256 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-65cc8b69-ec00-45dc-8777-f4a6400a3284 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dbd749e7fce4a19839deccdad621f68', 'request_id': 'req-65cc8b69-ec00-45dc-8777-f4a6400a3284'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:15.183185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-65cc8b69-ec00-45dc-8777-f4a6400a3284 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dbd749e7fce4a19839deccdad621f68', 'request_id': 'req-65cc8b69-ec00-45dc-8777-f4a6400a3284'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:15.190164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-65cc8b69-ec00-45dc-8777-f4a6400a3284 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dbd749e7fce4a19839deccdad621f68', 'request_id': 'req-65cc8b69-ec00-45dc-8777-f4a6400a3284'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:15.203609 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-65cc8b69-ec00-45dc-8777-f4a6400a3284 admin admin] The token cache is being invalidate because project 6dbd749e7fce4a19839deccdad621f68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:46:15.203926 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-65cc8b69-ec00-45dc-8777-f4a6400a3284 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:15.204604 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-65cc8b69-ec00-45dc-8777-f4a6400a3284 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:15.206109 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1346/2687] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:46:15 2022] DELETE /identity/v3/projects/6dbd749e7fce4a19839deccdad621f68 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:46:15.211230 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72735cd5-0b18-42f2-bcc5-d31ef1b3c88d None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:15.211685 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72735cd5-0b18-42f2-bcc5-d31ef1b3c88d None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:15.211807 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72735cd5-0b18-42f2-bcc5-d31ef1b3c88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:15.229343 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-72735cd5-0b18-42f2-bcc5-d31ef1b3c88d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:46:15.230522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-72735cd5-0b18-42f2-bcc5-d31ef1b3c88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:15.265097 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:46:15.294425 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1342/2688] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:46:15.302565 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6facb-b042-4523-8383-f769360dbb62 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:15.303031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6facb-b042-4523-8383-f769360dbb62 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:15.303127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6facb-b042-4523-8383-f769360dbb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:15.321792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-13f6facb-b042-4523-8383-f769360dbb62 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:46:15.323011 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-13f6facb-b042-4523-8383-f769360dbb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:15.330993 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1347/2689] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:46:15.335554 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72735cd5-0b18-42f2-bcc5-d31ef1b3c88d None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:15.345215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72735cd5-0b18-42f2-bcc5-d31ef1b3c88d None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:15.349948 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5099cff-9dbc-4c4f-a4f3-6ab6264e4f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:15.350594 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5099cff-9dbc-4c4f-a4f3-6ab6264e4f06 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:15.350877 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5099cff-9dbc-4c4f-a4f3-6ab6264e4f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:15.351153 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5099cff-9dbc-4c4f-a4f3-6ab6264e4f06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:15.351893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5099cff-9dbc-4c4f-a4f3-6ab6264e4f06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:46:15.352202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5099cff-9dbc-4c4f-a4f3-6ab6264e4f06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:15.353804 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5099cff-9dbc-4c4f-a4f3-6ab6264e4f06 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:15.365750 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1343/2690] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:46:15.370006 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6facb-b042-4523-8383-f769360dbb62 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:15.379266 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6facb-b042-4523-8383-f769360dbb62 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:15.384661 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1da298-c8f7-4ad9-998d-7eb1a37f27f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:15.385319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1da298-c8f7-4ad9-998d-7eb1a37f27f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:15.385599 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1da298-c8f7-4ad9-998d-7eb1a37f27f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:15.385875 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1da298-c8f7-4ad9-998d-7eb1a37f27f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:15.386599 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1da298-c8f7-4ad9-998d-7eb1a37f27f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:15.388827 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1da298-c8f7-4ad9-998d-7eb1a37f27f7 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:15.414535 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1348/2691] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:46:15 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:46:15.418686 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5099cff-9dbc-4c4f-a4f3-6ab6264e4f06 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:15.429107 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5099cff-9dbc-4c4f-a4f3-6ab6264e4f06 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:15.484285 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5e676-3191-4ef6-8a0c-cd632e1d864c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:15.484952 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5e676-3191-4ef6-8a0c-cd632e1d864c admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:15.485240 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5e676-3191-4ef6-8a0c-cd632e1d864c admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:15.485518 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5e676-3191-4ef6-8a0c-cd632e1d864c admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:15.486194 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5e676-3191-4ef6-8a0c-cd632e1d864c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:15.488138 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5e676-3191-4ef6-8a0c-cd632e1d864c admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:15.508951 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1344/2692] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:46:15 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:46:15.513258 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1da298-c8f7-4ad9-998d-7eb1a37f27f7 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:15.523544 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1da298-c8f7-4ad9-998d-7eb1a37f27f7 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:15.528137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c30da-9547-4459-9069-c97c4f1dff2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:15.528789 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c30da-9547-4459-9069-c97c4f1dff2f admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:15.529072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c30da-9547-4459-9069-c97c4f1dff2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:15.529347 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c30da-9547-4459-9069-c97c4f1dff2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:15.530094 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c30da-9547-4459-9069-c97c4f1dff2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:15.531674 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c30da-9547-4459-9069-c97c4f1dff2f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:15.542433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1349/2693] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:46:15.546561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5e676-3191-4ef6-8a0c-cd632e1d864c admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:15.557174 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5e676-3191-4ef6-8a0c-cd632e1d864c admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:15.561796 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93135ad-5524-48a6-8ddd-c52e0f2adf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:15.562444 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93135ad-5524-48a6-8ddd-c52e0f2adf32 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:15.562730 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93135ad-5524-48a6-8ddd-c52e0f2adf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:15.563008 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93135ad-5524-48a6-8ddd-c52e0f2adf32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:15.563728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93135ad-5524-48a6-8ddd-c52e0f2adf32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:15.565339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93135ad-5524-48a6-8ddd-c52e0f2adf32 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:15.575247 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1345/2694] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:46:15.579096 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c30da-9547-4459-9069-c97c4f1dff2f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:15.588776 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c30da-9547-4459-9069-c97c4f1dff2f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:15.593373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be8290-e5ea-46a6-bca3-004e94af7061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:15.594233 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be8290-e5ea-46a6-bca3-004e94af7061 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:15.594515 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be8290-e5ea-46a6-bca3-004e94af7061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:15.594790 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be8290-e5ea-46a6-bca3-004e94af7061 admin admin] PATH_INFO: `/v3/projects/ff9dc2c43c644142aa43d07783b49461/users/f726eccaf8c841bab02c664a93894bb8/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:15.609794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be8290-e5ea-46a6-bca3-004e94af7061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff9dc2c43c644142aa43d07783b49461, user_id=f726eccaf8c841bab02c664a93894bb8, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:15.612463 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be8290-e5ea-46a6-bca3-004e94af7061 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:15.626386 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1350/2695] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:46:15 2022] PUT /identity/v3/projects/ff9dc2c43c644142aa43d07783b49461/users/f726eccaf8c841bab02c664a93894bb8/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:46:15.636857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f8a82-11fc-4fcb-a4b8-d9608e8d7e7c None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:15.637401 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f8a82-11fc-4fcb-a4b8-d9608e8d7e7c None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:15.637709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f8a82-11fc-4fcb-a4b8-d9608e8d7e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:15.664503 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-fc3f8a82-11fc-4fcb-a4b8-d9608e8d7e7c None None] MFA Rules not processed for user `f726eccaf8c841bab02c664a93894bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:46:15.665707 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-fc3f8a82-11fc-4fcb-a4b8-d9608e8d7e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:15.729320 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1346/2696] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:46:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:46:15.740507 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be8290-e5ea-46a6-bca3-004e94af7061 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:15.742257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-11be8290-e5ea-46a6-bca3-004e94af7061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:15.798065 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be8290-e5ea-46a6-bca3-004e94af7061 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:15.802072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b2d87f-9fc5-44f4-b0cf-4d0e1af1a2f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:15.802722 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2d87f-9fc5-44f4-b0cf-4d0e1af1a2f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:15.803024 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2d87f-9fc5-44f4-b0cf-4d0e1af1a2f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:15.803281 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2d87f-9fc5-44f4-b0cf-4d0e1af1a2f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:15.811746 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b2d87f-9fc5-44f4-b0cf-4d0e1af1a2f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:15.813191 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b2d87f-9fc5-44f4-b0cf-4d0e1af1a2f7 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:15.822052 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1351/2697] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:46:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:46:25.726365 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:46:25.818124 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=60,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:46:28.844293 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f8a82-11fc-4fcb-a4b8-d9608e8d7e7c None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:28.905240 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f8a82-11fc-4fcb-a4b8-d9608e8d7e7c None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:28.909657 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912ba1f-ea17-474f-a058-1d140bbc48b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:28.910566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912ba1f-ea17-474f-a058-1d140bbc48b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:28.910848 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912ba1f-ea17-474f-a058-1d140bbc48b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:28.911118 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912ba1f-ea17-474f-a058-1d140bbc48b6 admin admin] PATH_INFO: `/v3/users/f726eccaf8c841bab02c664a93894bb8` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:28.912318 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912ba1f-ea17-474f-a058-1d140bbc48b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f726eccaf8c841bab02c664a93894bb8)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:28.914198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912ba1f-ea17-474f-a058-1d140bbc48b6 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:28.962517 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-c912ba1f-ea17-474f-a058-1d140bbc48b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f726eccaf8c841bab02c664a93894bb8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:28.969205 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-c912ba1f-ea17-474f-a058-1d140bbc48b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f726eccaf8c841bab02c664a93894bb8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:28.976754 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-c912ba1f-ea17-474f-a058-1d140bbc48b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f726eccaf8c841bab02c664a93894bb8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:28.977348 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-c912ba1f-ea17-474f-a058-1d140bbc48b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f726eccaf8c841bab02c664a93894bb8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:28.989804 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-c912ba1f-ea17-474f-a058-1d140bbc48b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f726eccaf8c841bab02c664a93894bb8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:28.995607 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1347/2698] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:46:28 2022] DELETE /identity/v3/users/f726eccaf8c841bab02c664a93894bb8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:46:29.000236 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3cb79-9996-41c8-9fae-546ce1e59beb None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:29.000742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3cb79-9996-41c8-9fae-546ce1e59beb None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:29.000847 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3cb79-9996-41c8-9fae-546ce1e59beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:29.019460 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-28d3cb79-9996-41c8-9fae-546ce1e59beb None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:46:29.020811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-28d3cb79-9996-41c8-9fae-546ce1e59beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:29.085651 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1352/2699] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:46:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:46:29.098514 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912ba1f-ea17-474f-a058-1d140bbc48b6 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:29.100076 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-c912ba1f-ea17-474f-a058-1d140bbc48b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:29.159590 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912ba1f-ea17-474f-a058-1d140bbc48b6 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:29.163539 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413d186-e54e-447d-97c7-6b15dc458f7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:29.164284 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413d186-e54e-447d-97c7-6b15dc458f7e service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:29.164594 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413d186-e54e-447d-97c7-6b15dc458f7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:29.164915 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413d186-e54e-447d-97c7-6b15dc458f7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:29.175254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413d186-e54e-447d-97c7-6b15dc458f7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:29.177316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413d186-e54e-447d-97c7-6b15dc458f7e service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:29.186839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1348/2700] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:46:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:46:29.453613 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3cb79-9996-41c8-9fae-546ce1e59beb None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:29.455135 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-28d3cb79-9996-41c8-9fae-546ce1e59beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:29.466871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3cb79-9996-41c8-9fae-546ce1e59beb None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:29.471582 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192dc9c-0562-48b7-a13c-ccc108d636e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:29.473045 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192dc9c-0562-48b7-a13c-ccc108d636e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:29.473326 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192dc9c-0562-48b7-a13c-ccc108d636e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:29.473602 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192dc9c-0562-48b7-a13c-ccc108d636e7 admin admin] PATH_INFO: `/v3/projects/ff9dc2c43c644142aa43d07783b49461` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:29.474793 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d192dc9c-0562-48b7-a13c-ccc108d636e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff9dc2c43c644142aa43d07783b49461)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:29.476778 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d192dc9c-0562-48b7-a13c-ccc108d636e7 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:29.526448 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d192dc9c-0562-48b7-a13c-ccc108d636e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff9dc2c43c644142aa43d07783b49461', 'request_id': 'req-d192dc9c-0562-48b7-a13c-ccc108d636e7'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:29.533540 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d192dc9c-0562-48b7-a13c-ccc108d636e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff9dc2c43c644142aa43d07783b49461', 'request_id': 'req-d192dc9c-0562-48b7-a13c-ccc108d636e7'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:29.539730 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d192dc9c-0562-48b7-a13c-ccc108d636e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff9dc2c43c644142aa43d07783b49461', 'request_id': 'req-d192dc9c-0562-48b7-a13c-ccc108d636e7'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:29.551074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d192dc9c-0562-48b7-a13c-ccc108d636e7 admin admin] The token cache is being invalidate because project ff9dc2c43c644142aa43d07783b49461 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:46:29.551501 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d192dc9c-0562-48b7-a13c-ccc108d636e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:29.552164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d192dc9c-0562-48b7-a13c-ccc108d636e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:29.553711 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1353/2701] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:46:29 2022] DELETE /identity/v3/projects/ff9dc2c43c644142aa43d07783b49461 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:46:29.558537 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa95a4-2b9b-4f20-bcdb-c5f5c88a8825 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:29.558830 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa95a4-2b9b-4f20-bcdb-c5f5c88a8825 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:29.559107 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa95a4-2b9b-4f20-bcdb-c5f5c88a8825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:29.576442 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-acaa95a4-2b9b-4f20-bcdb-c5f5c88a8825 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:46:29.577640 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-acaa95a4-2b9b-4f20-bcdb-c5f5c88a8825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:29.594071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:46:29.639445 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1349/2702] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:46:29.699727 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661f685-599b-48c2-94a3-b084f449dce7 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:29.700043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661f685-599b-48c2-94a3-b084f449dce7 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:29.700323 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661f685-599b-48c2-94a3-b084f449dce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:29.719865 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-0661f685-599b-48c2-94a3-b084f449dce7 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:46:29.721074 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0661f685-599b-48c2-94a3-b084f449dce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:29.729561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1354/2703] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:46:29.733821 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa95a4-2b9b-4f20-bcdb-c5f5c88a8825 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:29.743317 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa95a4-2b9b-4f20-bcdb-c5f5c88a8825 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:29.748160 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c61182-4b9f-46fe-85b4-03ea70e82ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:29.748840 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c61182-4b9f-46fe-85b4-03ea70e82ff5 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:29.749119 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c61182-4b9f-46fe-85b4-03ea70e82ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:29.749388 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c61182-4b9f-46fe-85b4-03ea70e82ff5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:29.750172 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c61182-4b9f-46fe-85b4-03ea70e82ff5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:46:29.750472 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c61182-4b9f-46fe-85b4-03ea70e82ff5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:29.752106 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c61182-4b9f-46fe-85b4-03ea70e82ff5 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:29.763939 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1350/2704] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:46:29.768090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661f685-599b-48c2-94a3-b084f449dce7 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:29.777852 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661f685-599b-48c2-94a3-b084f449dce7 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:29.782524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637648b-c507-4d4a-8777-767ee647c9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:29.783161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637648b-c507-4d4a-8777-767ee647c9c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:29.783445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637648b-c507-4d4a-8777-767ee647c9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:29.783720 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637648b-c507-4d4a-8777-767ee647c9c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:29.784439 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6637648b-c507-4d4a-8777-767ee647c9c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:29.786976 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6637648b-c507-4d4a-8777-767ee647c9c1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:29.812777 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1355/2705] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:46:29 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:46:29.816943 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c61182-4b9f-46fe-85b4-03ea70e82ff5 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:29.826956 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c61182-4b9f-46fe-85b4-03ea70e82ff5 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:29.882922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909103c3-81d7-48a7-a4e8-c90890c2536c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:29.883546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909103c3-81d7-48a7-a4e8-c90890c2536c admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:29.883814 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909103c3-81d7-48a7-a4e8-c90890c2536c admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:29.884089 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909103c3-81d7-48a7-a4e8-c90890c2536c admin admin] PATH_INFO: `/v3/users` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:29.884778 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909103c3-81d7-48a7-a4e8-c90890c2536c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:29.886931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909103c3-81d7-48a7-a4e8-c90890c2536c admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:29.906217 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1351/2706] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:46:29 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:46:29.910375 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637648b-c507-4d4a-8777-767ee647c9c1 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:29.920729 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637648b-c507-4d4a-8777-767ee647c9c1 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:29.925976 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b4005-d10b-43ed-8531-113d4a25a1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:29.926852 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b4005-d10b-43ed-8531-113d4a25a1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:29.927137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b4005-d10b-43ed-8531-113d4a25a1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:29.927415 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b4005-d10b-43ed-8531-113d4a25a1d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:29.928175 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b4005-d10b-43ed-8531-113d4a25a1d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:29.929764 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b4005-d10b-43ed-8531-113d4a25a1d5 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:29.939803 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1356/2707] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:46:29.944305 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909103c3-81d7-48a7-a4e8-c90890c2536c admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:29.954107 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909103c3-81d7-48a7-a4e8-c90890c2536c admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:29.958798 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04145d-997b-4f38-80cc-b9d1f794811f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:29.959446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04145d-997b-4f38-80cc-b9d1f794811f admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:29.959731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04145d-997b-4f38-80cc-b9d1f794811f admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:29.960042 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04145d-997b-4f38-80cc-b9d1f794811f admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:29.960768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04145d-997b-4f38-80cc-b9d1f794811f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:29.962336 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04145d-997b-4f38-80cc-b9d1f794811f admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:29.972519 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1352/2708] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:46:29.976332 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b4005-d10b-43ed-8531-113d4a25a1d5 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:29.989668 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b4005-d10b-43ed-8531-113d4a25a1d5 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:29.995314 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a58c0f-9a80-4286-aa20-2ab1f35795a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:29.996248 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a58c0f-9a80-4286-aa20-2ab1f35795a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:29.996537 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a58c0f-9a80-4286-aa20-2ab1f35795a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:29.996817 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a58c0f-9a80-4286-aa20-2ab1f35795a6 admin admin] PATH_INFO: `/v3/projects/bce7334f1f35405d88dafe998dc5168c/users/37e3191be5fa4c78984a51c936268e00/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:30.012426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a58c0f-9a80-4286-aa20-2ab1f35795a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bce7334f1f35405d88dafe998dc5168c, user_id=37e3191be5fa4c78984a51c936268e00, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:30.015049 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a58c0f-9a80-4286-aa20-2ab1f35795a6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:30.029130 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1357/2709] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:46:29 2022] PUT /identity/v3/projects/bce7334f1f35405d88dafe998dc5168c/users/37e3191be5fa4c78984a51c936268e00/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:46:30.040093 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf06f4-345a-4459-83dc-59427b8a9a78 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:30.040388 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf06f4-345a-4459-83dc-59427b8a9a78 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:30.040669 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf06f4-345a-4459-83dc-59427b8a9a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:30.067465 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-ebbf06f4-345a-4459-83dc-59427b8a9a78 None None] MFA Rules not processed for user `37e3191be5fa4c78984a51c936268e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:46:30.068711 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ebbf06f4-345a-4459-83dc-59427b8a9a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:30.132334 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1353/2710] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:46:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:46:30.143840 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a58c0f-9a80-4286-aa20-2ab1f35795a6 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:30.147312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-99a58c0f-9a80-4286-aa20-2ab1f35795a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:30.170110 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:46:30.209150 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a58c0f-9a80-4286-aa20-2ab1f35795a6 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:30.212854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17367348-85fb-4a7b-a18f-1360edc7bedd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:30.213494 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17367348-85fb-4a7b-a18f-1360edc7bedd service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:30.213775 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17367348-85fb-4a7b-a18f-1360edc7bedd service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:30.214050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17367348-85fb-4a7b-a18f-1360edc7bedd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:30.223946 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17367348-85fb-4a7b-a18f-1360edc7bedd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:30.225412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17367348-85fb-4a7b-a18f-1360edc7bedd service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:30.234115 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1358/2711] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:46:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:46:31.589052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad838e3-caaa-434d-a9c8-46321047fc58 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:31.589531 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad838e3-caaa-434d-a9c8-46321047fc58 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:31.589642 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad838e3-caaa-434d-a9c8-46321047fc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:31.607253 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-dad838e3-caaa-434d-a9c8-46321047fc58 None None] MFA Rules not processed for user `37e3191be5fa4c78984a51c936268e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:46:31.608570 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-dad838e3-caaa-434d-a9c8-46321047fc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:31.616272 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1354/2712] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:46:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:46:40.230129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=17 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:46:41.602188 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:46:50.970535 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b8cd8-4b78-4e59-b643-e31f8b4a3c68 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:50.971116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b8cd8-4b78-4e59-b643-e31f8b4a3c68 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:50.971852 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b8cd8-4b78-4e59-b643-e31f8b4a3c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:50.988844 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-dc4b8cd8-4b78-4e59-b643-e31f8b4a3c68 None None] MFA Rules not processed for user `e9e358afe0974c0c8bd53104ebbff445`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:46:50.990037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-dc4b8cd8-4b78-4e59-b643-e31f8b4a3c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:51.044745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1359/2713] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jun 6 10:46:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:46:51.093574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad838e3-caaa-434d-a9c8-46321047fc58 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:51.094942 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-dad838e3-caaa-434d-a9c8-46321047fc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:51.163804 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad838e3-caaa-434d-a9c8-46321047fc58 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:51.168774 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2224b-0dc2-47b7-bbfa-61cf6ea875c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fede9e52ad274285b09d748a684ede38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:51.169416 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2224b-0dc2-47b7-bbfa-61cf6ea875c9 service swift] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:51.169698 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2224b-0dc2-47b7-bbfa-61cf6ea875c9 service swift] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:51.169971 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2224b-0dc2-47b7-bbfa-61cf6ea875c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:51.178105 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2224b-0dc2-47b7-bbfa-61cf6ea875c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:51.179669 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2224b-0dc2-47b7-bbfa-61cf6ea875c9 service swift] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:51.187560 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1355/2714] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jun 6 10:46:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 10:46:51.701529 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44a9cb-dbde-4ee8-903a-11228103d9dc None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:51.701822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44a9cb-dbde-4ee8-903a-11228103d9dc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:51.702103 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44a9cb-dbde-4ee8-903a-11228103d9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:51.720091 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2c44a9cb-dbde-4ee8-903a-11228103d9dc None None] MFA Rules not processed for user `e9e358afe0974c0c8bd53104ebbff445`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:46:51.722106 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2c44a9cb-dbde-4ee8-903a-11228103d9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:51.729224 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1360/2715] 10.222.0.42 () {58 vars in 967 bytes} [Mon Jun 6 10:46:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:46:51.741474 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2224b-0dc2-47b7-bbfa-61cf6ea875c9 service swift] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:51.752731 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2224b-0dc2-47b7-bbfa-61cf6ea875c9 service swift] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:51.758263 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b315b7-eee0-4259-8e21-64b365e5d8ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fede9e52ad274285b09d748a684ede38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:51.758962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b315b7-eee0-4259-8e21-64b365e5d8ec service swift] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:51.759268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b315b7-eee0-4259-8e21-64b365e5d8ec service swift] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:51.759546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b315b7-eee0-4259-8e21-64b365e5d8ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:51.769026 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b315b7-eee0-4259-8e21-64b365e5d8ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:51.770544 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b315b7-eee0-4259-8e21-64b365e5d8ec service swift] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:51.779042 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1356/2716] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jun 6 10:46:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 10:46:51.966784 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2433b4-11e8-4c63-85aa-058b78a6972f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:51.967099 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2433b4-11e8-4c63-85aa-058b78a6972f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:51.967379 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2433b4-11e8-4c63-85aa-058b78a6972f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:51.985641 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-6b2433b4-11e8-4c63-85aa-058b78a6972f None None] MFA Rules not processed for user `e9e358afe0974c0c8bd53104ebbff445`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:46:51.986949 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6b2433b4-11e8-4c63-85aa-058b78a6972f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:51.994369 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1361/2717] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jun 6 10:46:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:46:52.006489 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b315b7-eee0-4259-8e21-64b365e5d8ec service swift] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:52.017644 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b315b7-eee0-4259-8e21-64b365e5d8ec service swift] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:52.023703 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18b408-77ce-488e-856f-e99b0b96c853 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fede9e52ad274285b09d748a684ede38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:52.024783 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18b408-77ce-488e-856f-e99b0b96c853 service swift] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:52.025296 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18b408-77ce-488e-856f-e99b0b96c853 service swift] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:52.025843 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18b408-77ce-488e-856f-e99b0b96c853 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:52.035052 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c18b408-77ce-488e-856f-e99b0b96c853 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:52.036735 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c18b408-77ce-488e-856f-e99b0b96c853 service swift] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:52.045296 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1357/2718] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jun 6 10:46:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 10:46:52.365564 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baea7bf-540b-457e-8de6-fd81f2314349 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:52.365963 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baea7bf-540b-457e-8de6-fd81f2314349 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:52.366275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baea7bf-540b-457e-8de6-fd81f2314349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:52.385983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9baea7bf-540b-457e-8de6-fd81f2314349 None None] MFA Rules not processed for user `e9e358afe0974c0c8bd53104ebbff445`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:46:52.387467 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9baea7bf-540b-457e-8de6-fd81f2314349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:52.395128 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1362/2719] 10.222.0.42 () {58 vars in 967 bytes} [Mon Jun 6 10:46:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:46:52.407076 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18b408-77ce-488e-856f-e99b0b96c853 service swift] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:52.417667 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18b408-77ce-488e-856f-e99b0b96c853 service swift] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:52.422612 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d07fa-8e52-4099-9b7a-1ba924e4c2d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fede9e52ad274285b09d748a684ede38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:52.423242 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d07fa-8e52-4099-9b7a-1ba924e4c2d6 service swift] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:52.423527 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d07fa-8e52-4099-9b7a-1ba924e4c2d6 service swift] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:52.423832 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d07fa-8e52-4099-9b7a-1ba924e4c2d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:52.432633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d07fa-8e52-4099-9b7a-1ba924e4c2d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:52.434228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d07fa-8e52-4099-9b7a-1ba924e4c2d6 service swift] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:52.443339 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1358/2720] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jun 6 10:46:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 10:46:54.491677 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baea7bf-540b-457e-8de6-fd81f2314349 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:54.556583 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baea7bf-540b-457e-8de6-fd81f2314349 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:54.561011 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ac211-1ab0-4cc7-8c41-0bc6b044acba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:54.561873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ac211-1ab0-4cc7-8c41-0bc6b044acba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:54.562155 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ac211-1ab0-4cc7-8c41-0bc6b044acba admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:54.562424 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ac211-1ab0-4cc7-8c41-0bc6b044acba admin admin] PATH_INFO: `/v3/users/37e3191be5fa4c78984a51c936268e00` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:54.563576 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ac211-1ab0-4cc7-8c41-0bc6b044acba admin admin] RBAC: Authorizing `identity:delete_user(user_id=37e3191be5fa4c78984a51c936268e00)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:54.566168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ac211-1ab0-4cc7-8c41-0bc6b044acba admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:54.610333 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-965ac211-1ab0-4cc7-8c41-0bc6b044acba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37e3191be5fa4c78984a51c936268e00'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:54.617537 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-965ac211-1ab0-4cc7-8c41-0bc6b044acba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37e3191be5fa4c78984a51c936268e00'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:54.618185 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-965ac211-1ab0-4cc7-8c41-0bc6b044acba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37e3191be5fa4c78984a51c936268e00'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:54.618779 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-965ac211-1ab0-4cc7-8c41-0bc6b044acba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37e3191be5fa4c78984a51c936268e00'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:54.630935 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-965ac211-1ab0-4cc7-8c41-0bc6b044acba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37e3191be5fa4c78984a51c936268e00'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:54.644395 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1363/2721] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:46:54 2022] DELETE /identity/v3/users/37e3191be5fa4c78984a51c936268e00 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:46:54.648904 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d4e75-25a8-40c3-b5c0-9bff07015417 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:54.649409 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d4e75-25a8-40c3-b5c0-9bff07015417 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:54.649508 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d4e75-25a8-40c3-b5c0-9bff07015417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:54.666789 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-bf8d4e75-25a8-40c3-b5c0-9bff07015417 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:46:54.668012 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-bf8d4e75-25a8-40c3-b5c0-9bff07015417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:54.733127 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1359/2722] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:46:54.743404 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ac211-1ab0-4cc7-8c41-0bc6b044acba admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:54.744850 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-965ac211-1ab0-4cc7-8c41-0bc6b044acba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:54.751258 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:46:54.805696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ac211-1ab0-4cc7-8c41-0bc6b044acba admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:54.809412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1d6fc-4eb1-494c-babc-95ada95126d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:54.810086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1d6fc-4eb1-494c-babc-95ada95126d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:54.810346 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1d6fc-4eb1-494c-babc-95ada95126d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:54.810681 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1d6fc-4eb1-494c-babc-95ada95126d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:54.820426 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1d6fc-4eb1-494c-babc-95ada95126d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:54.821862 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1d6fc-4eb1-494c-babc-95ada95126d4 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:54.831364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1364/2723] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:46:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:46:55.003886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d4e75-25a8-40c3-b5c0-9bff07015417 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:55.005776 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-bf8d4e75-25a8-40c3-b5c0-9bff07015417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:55.021132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d4e75-25a8-40c3-b5c0-9bff07015417 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:55.026288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c369cf-1bcc-4a12-be6f-aa65419f99c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:55.027198 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c369cf-1bcc-4a12-be6f-aa65419f99c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:55.027427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c369cf-1bcc-4a12-be6f-aa65419f99c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:55.027705 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c369cf-1bcc-4a12-be6f-aa65419f99c5 admin admin] PATH_INFO: `/v3/projects/bce7334f1f35405d88dafe998dc5168c` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:55.029168 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c369cf-1bcc-4a12-be6f-aa65419f99c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bce7334f1f35405d88dafe998dc5168c)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:55.031460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c369cf-1bcc-4a12-be6f-aa65419f99c5 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:55.094058 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:46:55.098070 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d0c369cf-1bcc-4a12-be6f-aa65419f99c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bce7334f1f35405d88dafe998dc5168c', 'request_id': 'req-d0c369cf-1bcc-4a12-be6f-aa65419f99c5'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:55.105422 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d0c369cf-1bcc-4a12-be6f-aa65419f99c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bce7334f1f35405d88dafe998dc5168c', 'request_id': 'req-d0c369cf-1bcc-4a12-be6f-aa65419f99c5'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:55.113532 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d0c369cf-1bcc-4a12-be6f-aa65419f99c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bce7334f1f35405d88dafe998dc5168c', 'request_id': 'req-d0c369cf-1bcc-4a12-be6f-aa65419f99c5'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:55.124666 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d0c369cf-1bcc-4a12-be6f-aa65419f99c5 admin admin] The token cache is being invalidate because project bce7334f1f35405d88dafe998dc5168c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:46:55.124952 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d0c369cf-1bcc-4a12-be6f-aa65419f99c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:55.125588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-d0c369cf-1bcc-4a12-be6f-aa65419f99c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:55.127129 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1360/2724] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:46:55 2022] DELETE /identity/v3/projects/bce7334f1f35405d88dafe998dc5168c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:46:55.132276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803adc3e-5f7a-479f-913d-262414ec71a9 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:55.132562 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803adc3e-5f7a-479f-913d-262414ec71a9 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:55.132840 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803adc3e-5f7a-479f-913d-262414ec71a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:55.152598 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-803adc3e-5f7a-479f-913d-262414ec71a9 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:46:55.153799 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-803adc3e-5f7a-479f-913d-262414ec71a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:55.218259 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1365/2725] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:46:55.226941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c4b0d-9130-404a-af29-035de0bcf3ff None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:55.227233 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c4b0d-9130-404a-af29-035de0bcf3ff None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:55.227513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c4b0d-9130-404a-af29-035de0bcf3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:55.246659 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-945c4b0d-9130-404a-af29-035de0bcf3ff None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:46:55.247952 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-945c4b0d-9130-404a-af29-035de0bcf3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:55.257188 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1361/2726] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:46:55.261317 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803adc3e-5f7a-479f-913d-262414ec71a9 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:55.271261 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803adc3e-5f7a-479f-913d-262414ec71a9 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:55.277147 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3637359-8754-499b-ace4-252f7b11e42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:55.277963 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3637359-8754-499b-ace4-252f7b11e42c admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:55.278257 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3637359-8754-499b-ace4-252f7b11e42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:55.278534 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3637359-8754-499b-ace4-252f7b11e42c admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:55.279256 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3637359-8754-499b-ace4-252f7b11e42c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:46:55.279560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3637359-8754-499b-ace4-252f7b11e42c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:55.281431 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3637359-8754-499b-ace4-252f7b11e42c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:55.294364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1366/2727] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:46:55.299638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c4b0d-9130-404a-af29-035de0bcf3ff None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:55.309909 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c4b0d-9130-404a-af29-035de0bcf3ff None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:55.315841 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc26c8-7702-4a81-9141-57c63399a02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:55.316551 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc26c8-7702-4a81-9141-57c63399a02e admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:55.316859 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc26c8-7702-4a81-9141-57c63399a02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:55.317459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc26c8-7702-4a81-9141-57c63399a02e admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:55.318128 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc26c8-7702-4a81-9141-57c63399a02e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:55.320510 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc26c8-7702-4a81-9141-57c63399a02e admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:55.357630 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1362/2728] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:46:55 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:46:55.362284 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3637359-8754-499b-ace4-252f7b11e42c admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:55.372666 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3637359-8754-499b-ace4-252f7b11e42c admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:55.429854 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367aa69-2e9f-4716-b501-4e8b5ba60666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:55.430434 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367aa69-2e9f-4716-b501-4e8b5ba60666 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:55.430717 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367aa69-2e9f-4716-b501-4e8b5ba60666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:55.430989 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367aa69-2e9f-4716-b501-4e8b5ba60666 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:55.431679 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1367aa69-2e9f-4716-b501-4e8b5ba60666 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:55.433599 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1367aa69-2e9f-4716-b501-4e8b5ba60666 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:55.454632 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1367/2729] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:46:55 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:46:55.458867 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc26c8-7702-4a81-9141-57c63399a02e admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:55.468918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc26c8-7702-4a81-9141-57c63399a02e admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:55.473456 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a836db-54a9-4720-a571-13c1fc0c5733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:55.474103 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a836db-54a9-4720-a571-13c1fc0c5733 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:55.474381 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a836db-54a9-4720-a571-13c1fc0c5733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:55.474653 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a836db-54a9-4720-a571-13c1fc0c5733 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:55.475362 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a836db-54a9-4720-a571-13c1fc0c5733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:55.476975 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a836db-54a9-4720-a571-13c1fc0c5733 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:55.487821 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1363/2730] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:46:55.491623 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367aa69-2e9f-4716-b501-4e8b5ba60666 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:55.501545 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367aa69-2e9f-4716-b501-4e8b5ba60666 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:55.506791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fae93d-ff85-4937-8018-f428626794ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:55.507433 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fae93d-ff85-4937-8018-f428626794ec admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:55.507714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fae93d-ff85-4937-8018-f428626794ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:55.508017 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fae93d-ff85-4937-8018-f428626794ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:55.508728 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fae93d-ff85-4937-8018-f428626794ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:55.510294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fae93d-ff85-4937-8018-f428626794ec admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:55.520460 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1368/2731] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:46:55.524398 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a836db-54a9-4720-a571-13c1fc0c5733 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:55.534794 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a836db-54a9-4720-a571-13c1fc0c5733 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:55.539363 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335b1f3-7836-4819-8fe4-a5436a52cba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:55.540271 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335b1f3-7836-4819-8fe4-a5436a52cba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:55.540556 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335b1f3-7836-4819-8fe4-a5436a52cba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:55.540837 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335b1f3-7836-4819-8fe4-a5436a52cba3 admin admin] PATH_INFO: `/v3/projects/6983c51eda664586ba62e206acfcc2d9/users/8a7a5a9773d84145bbf611439d025dd8/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:55.556217 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7335b1f3-7836-4819-8fe4-a5436a52cba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6983c51eda664586ba62e206acfcc2d9, user_id=8a7a5a9773d84145bbf611439d025dd8, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:55.558842 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7335b1f3-7836-4819-8fe4-a5436a52cba3 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:55.572103 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1364/2732] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:46:55 2022] PUT /identity/v3/projects/6983c51eda664586ba62e206acfcc2d9/users/8a7a5a9773d84145bbf611439d025dd8/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:46:55.577004 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738a912-7314-4907-99e6-d54b5ecd975c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:55.577318 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738a912-7314-4907-99e6-d54b5ecd975c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:55.577594 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738a912-7314-4907-99e6-d54b5ecd975c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:55.595243 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-8738a912-7314-4907-99e6-d54b5ecd975c None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:46:55.597244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-8738a912-7314-4907-99e6-d54b5ecd975c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:55.605303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:46:55.656292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1369/2733] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:46:55.666723 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335b1f3-7836-4819-8fe4-a5436a52cba3 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:55.668507 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-7335b1f3-7836-4819-8fe4-a5436a52cba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:55.727608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335b1f3-7836-4819-8fe4-a5436a52cba3 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:55.731454 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8242a9-d48b-4b1a-9ba1-2921637bec81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:55.732121 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8242a9-d48b-4b1a-9ba1-2921637bec81 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:55.732401 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8242a9-d48b-4b1a-9ba1-2921637bec81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:55.732673 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8242a9-d48b-4b1a-9ba1-2921637bec81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:55.743294 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8242a9-d48b-4b1a-9ba1-2921637bec81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:55.744735 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8242a9-d48b-4b1a-9ba1-2921637bec81 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:55.754954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1365/2734] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:46:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:46:57.908196 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738a912-7314-4907-99e6-d54b5ecd975c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:57.913252 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-8738a912-7314-4907-99e6-d54b5ecd975c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:57.921550 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8242a9-d48b-4b1a-9ba1-2921637bec81 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:57.925092 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ac8242a9-d48b-4b1a-9ba1-2921637bec81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:57.951562 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8242a9-d48b-4b1a-9ba1-2921637bec81 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:57.960941 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767173c3-dcab-4712-8f0d-32a2c4e241aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:57.966539 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767173c3-dcab-4712-8f0d-32a2c4e241aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:57.966539 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767173c3-dcab-4712-8f0d-32a2c4e241aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:57.966539 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767173c3-dcab-4712-8f0d-32a2c4e241aa admin admin] PATH_INFO: `/v3/users/63d02effcaa64a2580cdd1613e7ff1d8` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:57.966539 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767173c3-dcab-4712-8f0d-32a2c4e241aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=63d02effcaa64a2580cdd1613e7ff1d8)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:57.967508 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767173c3-dcab-4712-8f0d-32a2c4e241aa admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:58.056660 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738a912-7314-4907-99e6-d54b5ecd975c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:58.066417 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f519c79-0c3a-46b1-b76b-1a0e06ad8363 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361be69f1c0a41c18afd206bccf56447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:58.067567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f519c79-0c3a-46b1-b76b-1a0e06ad8363 service placement] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:58.068748 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f519c79-0c3a-46b1-b76b-1a0e06ad8363 service placement] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:58.069271 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f519c79-0c3a-46b1-b76b-1a0e06ad8363 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:58.073436 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1f519c79-0c3a-46b1-b76b-1a0e06ad8363 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:58.088221 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-767173c3-dcab-4712-8f0d-32a2c4e241aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63d02effcaa64a2580cdd1613e7ff1d8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:58.098756 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-767173c3-dcab-4712-8f0d-32a2c4e241aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63d02effcaa64a2580cdd1613e7ff1d8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:58.112886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-767173c3-dcab-4712-8f0d-32a2c4e241aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63d02effcaa64a2580cdd1613e7ff1d8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:58.117218 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-767173c3-dcab-4712-8f0d-32a2c4e241aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63d02effcaa64a2580cdd1613e7ff1d8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:58.117816 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f519c79-0c3a-46b1-b76b-1a0e06ad8363 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:58.119450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f519c79-0c3a-46b1-b76b-1a0e06ad8363 service placement] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:58.122673 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:DELETE=12,keystone:SELECT=84,keystone:INSERT=4 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:46:58.137489 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-767173c3-dcab-4712-8f0d-32a2c4e241aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63d02effcaa64a2580cdd1613e7ff1d8'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:58.150378 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1366/2735] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:46:57 2022] DELETE /identity/v3/users/63d02effcaa64a2580cdd1613e7ff1d8 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:46:58.150541 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1370/2736] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:46:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:46:58.444293 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767173c3-dcab-4712-8f0d-32a2c4e241aa admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:46:58.445747 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-767173c3-dcab-4712-8f0d-32a2c4e241aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:46:58.568768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767173c3-dcab-4712-8f0d-32a2c4e241aa admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:46:58.584309 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03dbd0d-ced8-473c-8f18-f860b02c779d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:46:58.585224 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03dbd0d-ced8-473c-8f18-f860b02c779d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:46:58.585515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03dbd0d-ced8-473c-8f18-f860b02c779d admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:46:58.585798 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03dbd0d-ced8-473c-8f18-f860b02c779d admin admin] PATH_INFO: `/v3/projects/d93c9242f08447aaa597423812d5904d` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:46:58.588894 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03dbd0d-ced8-473c-8f18-f860b02c779d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d93c9242f08447aaa597423812d5904d)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:46:58.590857 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03dbd0d-ced8-473c-8f18-f860b02c779d admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:46:58.688844 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e03dbd0d-ced8-473c-8f18-f860b02c779d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd93c9242f08447aaa597423812d5904d', 'request_id': 'req-e03dbd0d-ced8-473c-8f18-f860b02c779d'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:58.698162 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e03dbd0d-ced8-473c-8f18-f860b02c779d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd93c9242f08447aaa597423812d5904d', 'request_id': 'req-e03dbd0d-ced8-473c-8f18-f860b02c779d'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:58.712228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e03dbd0d-ced8-473c-8f18-f860b02c779d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd93c9242f08447aaa597423812d5904d', 'request_id': 'req-e03dbd0d-ced8-473c-8f18-f860b02c779d'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:58.731032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e03dbd0d-ced8-473c-8f18-f860b02c779d admin admin] The token cache is being invalidate because project d93c9242f08447aaa597423812d5904d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:46:58.732009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e03dbd0d-ced8-473c-8f18-f860b02c779d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:58.733066 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-e03dbd0d-ced8-473c-8f18-f860b02c779d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:46:58.759840 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1367/2737] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:46:58 2022] DELETE /identity/v3/projects/d93c9242f08447aaa597423812d5904d => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:47:00.595044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d503b3-2294-44d1-bef4-5ef301d78417 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:47:00.595599 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d503b3-2294-44d1-bef4-5ef301d78417 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:47:00.595883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d503b3-2294-44d1-bef4-5ef301d78417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:47:00.629290 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-26d503b3-2294-44d1-bef4-5ef301d78417 None None] MFA Rules not processed for user `8a7a5a9773d84145bbf611439d025dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:47:00.630525 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-26d503b3-2294-44d1-bef4-5ef301d78417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:47:00.700687 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1371/2738] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:47:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:47:00.705068 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03dbd0d-ced8-473c-8f18-f860b02c779d admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:47:00.707019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-e03dbd0d-ced8-473c-8f18-f860b02c779d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:47:00.775314 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03dbd0d-ced8-473c-8f18-f860b02c779d admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:47:00.780031 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64d4a2-14f7-403b-a104-b71dcfcadb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:47:00.780694 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64d4a2-14f7-403b-a104-b71dcfcadb6a admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:47:00.780979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64d4a2-14f7-403b-a104-b71dcfcadb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:47:00.781274 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64d4a2-14f7-403b-a104-b71dcfcadb6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:47:00.781975 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64d4a2-14f7-403b-a104-b71dcfcadb6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:47:00.784465 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64d4a2-14f7-403b-a104-b71dcfcadb6a admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:47:00.804763 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:47:00.813299 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1368/2739] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:47:00 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:47:00.818370 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d503b3-2294-44d1-bef4-5ef301d78417 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:47:00.842082 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d503b3-2294-44d1-bef4-5ef301d78417 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:47:00.888587 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:47:00.926944 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826526e9-2aa0-479d-8bf9-5dfa15089642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:47:00.927689 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826526e9-2aa0-479d-8bf9-5dfa15089642 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:47:00.928015 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826526e9-2aa0-479d-8bf9-5dfa15089642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:47:00.928306 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826526e9-2aa0-479d-8bf9-5dfa15089642 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:47:00.930211 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826526e9-2aa0-479d-8bf9-5dfa15089642 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:47:00.932315 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826526e9-2aa0-479d-8bf9-5dfa15089642 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:47:00.967913 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1372/2740] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:47:00 2022] POST /identity/v3/users => generated 296 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:47:00.971661 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64d4a2-14f7-403b-a104-b71dcfcadb6a admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:47:00.998080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64d4a2-14f7-403b-a104-b71dcfcadb6a admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:47:01.014071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a960a4-52d5-4326-81aa-3c1e3b81580b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:47:01.014767 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a960a4-52d5-4326-81aa-3c1e3b81580b admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:47:01.015066 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a960a4-52d5-4326-81aa-3c1e3b81580b admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:47:01.015352 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a960a4-52d5-4326-81aa-3c1e3b81580b admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:47:01.016816 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a960a4-52d5-4326-81aa-3c1e3b81580b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:47:01.023862 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a960a4-52d5-4326-81aa-3c1e3b81580b admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:47:01.037401 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1369/2741] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:47:01.041558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826526e9-2aa0-479d-8bf9-5dfa15089642 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:47:01.060026 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826526e9-2aa0-479d-8bf9-5dfa15089642 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:47:01.071636 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae35d5-9a0b-43ab-8097-6861fce8bd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:47:01.072968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae35d5-9a0b-43ab-8097-6861fce8bd73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:47:01.073115 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae35d5-9a0b-43ab-8097-6861fce8bd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:47:01.075410 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae35d5-9a0b-43ab-8097-6861fce8bd73 admin admin] PATH_INFO: `/v3/projects/9c36285723d74e1ea6ae42e57ad90b12/users/10b549e1e9fb4e34991fc90fc81d6f9e/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:47:01.107371 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae35d5-9a0b-43ab-8097-6861fce8bd73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c36285723d74e1ea6ae42e57ad90b12, user_id=10b549e1e9fb4e34991fc90fc81d6f9e, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:47:01.110499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae35d5-9a0b-43ab-8097-6861fce8bd73 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:47:01.131138 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1373/2742] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:47:01 2022] PUT /identity/v3/projects/9c36285723d74e1ea6ae42e57ad90b12/users/10b549e1e9fb4e34991fc90fc81d6f9e/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:47:01.133267 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a960a4-52d5-4326-81aa-3c1e3b81580b admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:47:01.157050 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a960a4-52d5-4326-81aa-3c1e3b81580b admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:47:01.236310 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586bdab-e794-4511-86f1-d029336787d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:47:01.237088 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586bdab-e794-4511-86f1-d029336787d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:47:01.237561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586bdab-e794-4511-86f1-d029336787d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:47:01.238216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586bdab-e794-4511-86f1-d029336787d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:47:01.239612 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586bdab-e794-4511-86f1-d029336787d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:47:01.241417 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586bdab-e794-4511-86f1-d029336787d5 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:47:01.254979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1370/2743] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:47:01.258511 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae35d5-9a0b-43ab-8097-6861fce8bd73 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:47:01.273158 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae35d5-9a0b-43ab-8097-6861fce8bd73 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:47:01.280809 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b3461-d28c-4880-be6d-138748c1d9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:47:01.281990 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b3461-d28c-4880-be6d-138748c1d9ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:47:01.282422 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b3461-d28c-4880-be6d-138748c1d9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:47:01.282608 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b3461-d28c-4880-be6d-138748c1d9ff admin admin] PATH_INFO: `/v3/projects/9c36285723d74e1ea6ae42e57ad90b12/users/10b549e1e9fb4e34991fc90fc81d6f9e/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:47:01.285302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b3461-d28c-4880-be6d-138748c1d9ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c36285723d74e1ea6ae42e57ad90b12, user_id=10b549e1e9fb4e34991fc90fc81d6f9e, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:47:01.295137 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b3461-d28c-4880-be6d-138748c1d9ff admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:47:01.313808 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1374/2744] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:47:01 2022] PUT /identity/v3/projects/9c36285723d74e1ea6ae42e57ad90b12/users/10b549e1e9fb4e34991fc90fc81d6f9e/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:47:05.765446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee8881-9c1d-4a64-bac6-a53988d26aaa None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:47:05.766680 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee8881-9c1d-4a64-bac6-a53988d26aaa None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:47:05.766977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee8881-9c1d-4a64-bac6-a53988d26aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:47:05.788623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-3fee8881-9c1d-4a64-bac6-a53988d26aaa None None] MFA Rules not processed for user `7ba341ea1ac34874bbfb8b2f713aea1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:47:05.790169 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-3fee8881-9c1d-4a64-bac6-a53988d26aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:47:05.910388 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1371/2745] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:47:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:47:06.817880 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331da33-df73-4e42-a0bb-ea649c25cb75 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:47:06.818483 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331da33-df73-4e42-a0bb-ea649c25cb75 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:47:06.818483 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331da33-df73-4e42-a0bb-ea649c25cb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:47:06.853707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3331da33-df73-4e42-a0bb-ea649c25cb75 None None] MFA Rules not processed for user `10b549e1e9fb4e34991fc90fc81d6f9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:47:06.854982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3331da33-df73-4e42-a0bb-ea649c25cb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:47:06.952788 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1375/2746] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:47:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:47:06.967533 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fee8881-9c1d-4a64-bac6-a53988d26aaa None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:47:06.969004 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-3fee8881-9c1d-4a64-bac6-a53988d26aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:47:07.079617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fee8881-9c1d-4a64-bac6-a53988d26aaa None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:47:07.089062 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e634306-116d-4814-afce-733e8c8d34f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:47:07.089901 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e634306-116d-4814-afce-733e8c8d34f3 service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:47:07.090216 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e634306-116d-4814-afce-733e8c8d34f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:47:07.090490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e634306-116d-4814-afce-733e8c8d34f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:47:07.138575 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:47:07.185481 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e634306-116d-4814-afce-733e8c8d34f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:47:07.187979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e634306-116d-4814-afce-733e8c8d34f3 service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:47:07.200101 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1372/2747] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:47:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:47:16.949558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=63,keystone:INSERT=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:47:17.194274 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=15 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:47:24.001189 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe15cf-7ec4-4d20-af9b-00c60b27f6b3 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:47:24.001685 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe15cf-7ec4-4d20-af9b-00c60b27f6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:47:24.001782 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe15cf-7ec4-4d20-af9b-00c60b27f6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:47:24.021053 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-0bbe15cf-7ec4-4d20-af9b-00c60b27f6b3 None None] MFA Rules not processed for user `8a7a5a9773d84145bbf611439d025dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:47:24.022252 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0bbe15cf-7ec4-4d20-af9b-00c60b27f6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:47:24.029037 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1376/2748] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:47:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:47:34.014757 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:47:41.279979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e634306-116d-4814-afce-733e8c8d34f3 service nova] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:47:41.281477 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-7e634306-116d-4814-afce-733e8c8d34f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:47:41.340217 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e634306-116d-4814-afce-733e8c8d34f3 service nova] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:47:41.343809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e32c55-822f-4586-8e0e-0d5581c7a071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:47:41.344454 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e32c55-822f-4586-8e0e-0d5581c7a071 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:47:41.344735 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e32c55-822f-4586-8e0e-0d5581c7a071 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:47:41.345009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e32c55-822f-4586-8e0e-0d5581c7a071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:47:41.354400 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e32c55-822f-4586-8e0e-0d5581c7a071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:47:41.355825 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e32c55-822f-4586-8e0e-0d5581c7a071 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:47:41.365595 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1373/2749] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:47:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:47:51.361054 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=26 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:10.540132 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe15cf-7ec4-4d20-af9b-00c60b27f6b3 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:10.542151 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0bbe15cf-7ec4-4d20-af9b-00c60b27f6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:10.615200 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe15cf-7ec4-4d20-af9b-00c60b27f6b3 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:10.619772 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de583b26-6007-4422-93b1-672e8873abb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:10.620730 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de583b26-6007-4422-93b1-672e8873abb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:10.621013 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de583b26-6007-4422-93b1-672e8873abb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:10.621293 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de583b26-6007-4422-93b1-672e8873abb6 admin admin] PATH_INFO: `/v3/users/7ba341ea1ac34874bbfb8b2f713aea1b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:10.622517 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de583b26-6007-4422-93b1-672e8873abb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ba341ea1ac34874bbfb8b2f713aea1b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:10.624498 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de583b26-6007-4422-93b1-672e8873abb6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:10.678299 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-de583b26-6007-4422-93b1-672e8873abb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ba341ea1ac34874bbfb8b2f713aea1b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:10.685214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-de583b26-6007-4422-93b1-672e8873abb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ba341ea1ac34874bbfb8b2f713aea1b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:10.685884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-de583b26-6007-4422-93b1-672e8873abb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ba341ea1ac34874bbfb8b2f713aea1b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:10.686484 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-de583b26-6007-4422-93b1-672e8873abb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ba341ea1ac34874bbfb8b2f713aea1b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:10.699062 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-de583b26-6007-4422-93b1-672e8873abb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ba341ea1ac34874bbfb8b2f713aea1b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:10.712524 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1377/2750] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:48:10 2022] DELETE /identity/v3/users/7ba341ea1ac34874bbfb8b2f713aea1b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:48:10.891167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e32c55-822f-4586-8e0e-0d5581c7a071 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:10.903022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e32c55-822f-4586-8e0e-0d5581c7a071 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:10.966942 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf204e0-9084-49f8-baad-3bfb42b0a870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:10.967827 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf204e0-9084-49f8-baad-3bfb42b0a870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:10.968110 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf204e0-9084-49f8-baad-3bfb42b0a870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:10.968381 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf204e0-9084-49f8-baad-3bfb42b0a870 admin admin] PATH_INFO: `/v3/projects/92244b4ae17d49bebe621e803bd9ae58` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:10.969526 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf204e0-9084-49f8-baad-3bfb42b0a870 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92244b4ae17d49bebe621e803bd9ae58)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:10.971701 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf204e0-9084-49f8-baad-3bfb42b0a870 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:11.019705 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-caf204e0-9084-49f8-baad-3bfb42b0a870 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92244b4ae17d49bebe621e803bd9ae58', 'request_id': 'req-caf204e0-9084-49f8-baad-3bfb42b0a870'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:11.029550 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-caf204e0-9084-49f8-baad-3bfb42b0a870 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92244b4ae17d49bebe621e803bd9ae58', 'request_id': 'req-caf204e0-9084-49f8-baad-3bfb42b0a870'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:11.037624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-caf204e0-9084-49f8-baad-3bfb42b0a870 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92244b4ae17d49bebe621e803bd9ae58', 'request_id': 'req-caf204e0-9084-49f8-baad-3bfb42b0a870'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:11.049210 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-caf204e0-9084-49f8-baad-3bfb42b0a870 admin admin] The token cache is being invalidate because project 92244b4ae17d49bebe621e803bd9ae58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:48:11.050919 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-caf204e0-9084-49f8-baad-3bfb42b0a870 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:11.051133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-caf204e0-9084-49f8-baad-3bfb42b0a870 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:11.052740 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1374/2751] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:48:10 2022] DELETE /identity/v3/projects/92244b4ae17d49bebe621e803bd9ae58 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:48:11.057767 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1569a43-7f30-40a4-8ecb-2ed06075d54e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:11.058061 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1569a43-7f30-40a4-8ecb-2ed06075d54e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:11.058373 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1569a43-7f30-40a4-8ecb-2ed06075d54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:11.076765 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c1569a43-7f30-40a4-8ecb-2ed06075d54e None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:11.078086 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c1569a43-7f30-40a4-8ecb-2ed06075d54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:11.138220 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1378/2752] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:11.146952 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cb0b9-cb9d-4d66-8735-0250789d4677 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:11.147240 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cb0b9-cb9d-4d66-8735-0250789d4677 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:11.147517 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cb0b9-cb9d-4d66-8735-0250789d4677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:11.166107 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-4e7cb0b9-cb9d-4d66-8735-0250789d4677 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:11.167312 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-4e7cb0b9-cb9d-4d66-8735-0250789d4677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:11.175250 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1375/2753] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:11.179389 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1569a43-7f30-40a4-8ecb-2ed06075d54e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:11.188983 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1569a43-7f30-40a4-8ecb-2ed06075d54e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:11.193768 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3704936-ad61-4d54-9cbf-1d6fe1a02683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:11.194414 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3704936-ad61-4d54-9cbf-1d6fe1a02683 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:11.194702 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3704936-ad61-4d54-9cbf-1d6fe1a02683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:11.194976 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3704936-ad61-4d54-9cbf-1d6fe1a02683 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:11.195708 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3704936-ad61-4d54-9cbf-1d6fe1a02683 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:48:11.196045 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3704936-ad61-4d54-9cbf-1d6fe1a02683 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:11.197638 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3704936-ad61-4d54-9cbf-1d6fe1a02683 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:11.212289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1379/2754] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:48:11.216667 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7cb0b9-cb9d-4d66-8735-0250789d4677 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:11.226905 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7cb0b9-cb9d-4d66-8735-0250789d4677 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:11.231576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c870198-4fc9-4a47-b4ce-2e8f978fd425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:11.232266 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c870198-4fc9-4a47-b4ce-2e8f978fd425 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:11.232549 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c870198-4fc9-4a47-b4ce-2e8f978fd425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:11.232823 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c870198-4fc9-4a47-b4ce-2e8f978fd425 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:11.233505 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c870198-4fc9-4a47-b4ce-2e8f978fd425 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:11.235389 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c870198-4fc9-4a47-b4ce-2e8f978fd425 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:11.261939 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1376/2755] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:48:11 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:48:11.266339 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3704936-ad61-4d54-9cbf-1d6fe1a02683 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:11.276944 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3704936-ad61-4d54-9cbf-1d6fe1a02683 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:11.280447 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:11.335217 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76007556-d7a0-4d52-9c30-96f976783a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:11.335836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76007556-d7a0-4d52-9c30-96f976783a85 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:11.336125 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76007556-d7a0-4d52-9c30-96f976783a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:11.336404 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76007556-d7a0-4d52-9c30-96f976783a85 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:11.337103 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76007556-d7a0-4d52-9c30-96f976783a85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:11.339039 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76007556-d7a0-4d52-9c30-96f976783a85 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:11.358161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1380/2756] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:48:11 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:48:11.363044 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c870198-4fc9-4a47-b4ce-2e8f978fd425 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:11.373134 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c870198-4fc9-4a47-b4ce-2e8f978fd425 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:11.377753 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111628b-5f54-4a99-bbfb-48569d904ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:11.378393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111628b-5f54-4a99-bbfb-48569d904ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:11.378677 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111628b-5f54-4a99-bbfb-48569d904ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:11.378953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111628b-5f54-4a99-bbfb-48569d904ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:11.379675 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111628b-5f54-4a99-bbfb-48569d904ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:11.381610 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111628b-5f54-4a99-bbfb-48569d904ba4 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:11.393239 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1377/2757] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:48:11.397289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76007556-d7a0-4d52-9c30-96f976783a85 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:11.407107 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76007556-d7a0-4d52-9c30-96f976783a85 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:11.412375 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32d92b-2921-4b17-bd5f-188e610a614b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:11.413023 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32d92b-2921-4b17-bd5f-188e610a614b admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:11.413305 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32d92b-2921-4b17-bd5f-188e610a614b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:11.413585 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32d92b-2921-4b17-bd5f-188e610a614b admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:11.414302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32d92b-2921-4b17-bd5f-188e610a614b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:11.415912 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32d92b-2921-4b17-bd5f-188e610a614b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:11.426445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1381/2758] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:48:11.430565 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111628b-5f54-4a99-bbfb-48569d904ba4 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:11.440306 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111628b-5f54-4a99-bbfb-48569d904ba4 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:11.445131 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c9972-85e4-4748-9cb4-d78ee06976b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:11.446091 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c9972-85e4-4748-9cb4-d78ee06976b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:11.446368 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c9972-85e4-4748-9cb4-d78ee06976b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:11.446638 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c9972-85e4-4748-9cb4-d78ee06976b7 admin admin] PATH_INFO: `/v3/projects/d648b3f0a1034e908d4977aa96586d4d/users/f005bfb957aa4adfa85f5e09c4c266fc/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:11.460923 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c9972-85e4-4748-9cb4-d78ee06976b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d648b3f0a1034e908d4977aa96586d4d, user_id=f005bfb957aa4adfa85f5e09c4c266fc, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:11.463507 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c9972-85e4-4748-9cb4-d78ee06976b7 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:11.476322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1378/2759] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:48:11 2022] PUT /identity/v3/projects/d648b3f0a1034e908d4977aa96586d4d/users/f005bfb957aa4adfa85f5e09c4c266fc/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:11.488048 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6338df2-4393-43a1-b799-0ff6b6ac6981 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:11.488357 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6338df2-4393-43a1-b799-0ff6b6ac6981 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:11.488634 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6338df2-4393-43a1-b799-0ff6b6ac6981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:11.514643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b6338df2-4393-43a1-b799-0ff6b6ac6981 None None] MFA Rules not processed for user `f005bfb957aa4adfa85f5e09c4c266fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:11.515894 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b6338df2-4393-43a1-b799-0ff6b6ac6981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:11.578251 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1382/2760] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:11.582988 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c9972-85e4-4748-9cb4-d78ee06976b7 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:11.613542 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:11.647288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c9972-85e4-4748-9cb4-d78ee06976b7 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:11.651906 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5655f4-c4c2-4579-b287-82c72ad3f809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:11.652552 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5655f4-c4c2-4579-b287-82c72ad3f809 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:11.652835 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5655f4-c4c2-4579-b287-82c72ad3f809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:11.653107 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5655f4-c4c2-4579-b287-82c72ad3f809 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:11.653794 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5655f4-c4c2-4579-b287-82c72ad3f809 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:11.655672 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5655f4-c4c2-4579-b287-82c72ad3f809 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:11.680746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1379/2761] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:48:11 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:48:11.684840 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6338df2-4393-43a1-b799-0ff6b6ac6981 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:11.695900 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6338df2-4393-43a1-b799-0ff6b6ac6981 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:11.750767 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22c49f-8a45-4dd0-afb5-8d2795872a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:11.751424 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22c49f-8a45-4dd0-afb5-8d2795872a97 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:11.751708 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22c49f-8a45-4dd0-afb5-8d2795872a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:11.752013 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22c49f-8a45-4dd0-afb5-8d2795872a97 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:11.752745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22c49f-8a45-4dd0-afb5-8d2795872a97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:11.754644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22c49f-8a45-4dd0-afb5-8d2795872a97 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:11.774414 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1383/2762] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:48:11 2022] POST /identity/v3/users => generated 296 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:48:11.778382 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5655f4-c4c2-4579-b287-82c72ad3f809 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:11.790012 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5655f4-c4c2-4579-b287-82c72ad3f809 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:11.794560 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcf719-fe14-4ccc-a518-0476482dd9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:11.795280 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcf719-fe14-4ccc-a518-0476482dd9fd admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:11.795541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcf719-fe14-4ccc-a518-0476482dd9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:11.795854 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcf719-fe14-4ccc-a518-0476482dd9fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:11.796567 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bcf719-fe14-4ccc-a518-0476482dd9fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:11.798141 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bcf719-fe14-4ccc-a518-0476482dd9fd admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:11.808543 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1380/2763] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:48:11.813062 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22c49f-8a45-4dd0-afb5-8d2795872a97 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:11.819398 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:11.823482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22c49f-8a45-4dd0-afb5-8d2795872a97 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:11.829159 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d15de-d6b5-4d01-b806-29e18181bf3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:11.830032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d15de-d6b5-4d01-b806-29e18181bf3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:11.830315 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d15de-d6b5-4d01-b806-29e18181bf3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:11.831080 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d15de-d6b5-4d01-b806-29e18181bf3c admin admin] PATH_INFO: `/v3/projects/789c62be7aa04e358983fe3ca2a0a122/users/4b02cddd36bd4f79bd4f63d489d5935b/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:11.845501 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d15de-d6b5-4d01-b806-29e18181bf3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=789c62be7aa04e358983fe3ca2a0a122, user_id=4b02cddd36bd4f79bd4f63d489d5935b, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:11.848917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d15de-d6b5-4d01-b806-29e18181bf3c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:11.861003 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1384/2764] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:48:11 2022] PUT /identity/v3/projects/789c62be7aa04e358983fe3ca2a0a122/users/4b02cddd36bd4f79bd4f63d489d5935b/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:11.864963 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcf719-fe14-4ccc-a518-0476482dd9fd admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:11.874762 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcf719-fe14-4ccc-a518-0476482dd9fd admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:11.931316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19873336-6abf-4fac-8fde-247705640589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:11.931990 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19873336-6abf-4fac-8fde-247705640589 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:11.932272 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19873336-6abf-4fac-8fde-247705640589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:11.932548 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19873336-6abf-4fac-8fde-247705640589 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:11.933274 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19873336-6abf-4fac-8fde-247705640589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:11.934822 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19873336-6abf-4fac-8fde-247705640589 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:11.944725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1381/2765] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:48:11.948826 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d15de-d6b5-4d01-b806-29e18181bf3c admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:11.959762 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d15de-d6b5-4d01-b806-29e18181bf3c admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:11.964419 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2dfb9-ed50-40ea-a114-ebc911a2d172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:11.965287 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2dfb9-ed50-40ea-a114-ebc911a2d172 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:11.965571 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2dfb9-ed50-40ea-a114-ebc911a2d172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:11.965846 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2dfb9-ed50-40ea-a114-ebc911a2d172 admin admin] PATH_INFO: `/v3/projects/789c62be7aa04e358983fe3ca2a0a122/users/4b02cddd36bd4f79bd4f63d489d5935b/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:11.967849 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2dfb9-ed50-40ea-a114-ebc911a2d172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=789c62be7aa04e358983fe3ca2a0a122, user_id=4b02cddd36bd4f79bd4f63d489d5935b, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:11.970446 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2dfb9-ed50-40ea-a114-ebc911a2d172 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:11.981141 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1385/2766] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:48:11 2022] PUT /identity/v3/projects/789c62be7aa04e358983fe3ca2a0a122/users/4b02cddd36bd4f79bd4f63d489d5935b/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:48:11.991189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a66e5-0ed5-445c-8cf9-dd8723ce427d None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:11.991532 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a66e5-0ed5-445c-8cf9-dd8723ce427d None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:11.991847 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a66e5-0ed5-445c-8cf9-dd8723ce427d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:12.019441 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-097a66e5-0ed5-445c-8cf9-dd8723ce427d None None] MFA Rules not processed for user `4b02cddd36bd4f79bd4f63d489d5935b`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:12.020690 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-097a66e5-0ed5-445c-8cf9-dd8723ce427d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:12.089114 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1382/2767] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:12.100366 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2dfb9-ed50-40ea-a114-ebc911a2d172 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:12.102073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fbe2dfb9-ed50-40ea-a114-ebc911a2d172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:12.159869 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2dfb9-ed50-40ea-a114-ebc911a2d172 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:12.163385 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0e64a-cc92-4d65-8dd0-b33ebf690ca6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:12.164045 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0e64a-cc92-4d65-8dd0-b33ebf690ca6 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:12.164326 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0e64a-cc92-4d65-8dd0-b33ebf690ca6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:12.164597 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0e64a-cc92-4d65-8dd0-b33ebf690ca6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:12.173754 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0e64a-cc92-4d65-8dd0-b33ebf690ca6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:12.175428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0e64a-cc92-4d65-8dd0-b33ebf690ca6 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:12.184993 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1386/2768] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:48:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:48:12.254667 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a66e5-0ed5-445c-8cf9-dd8723ce427d None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:12.264165 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a66e5-0ed5-445c-8cf9-dd8723ce427d None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:12.267972 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56d404-6ace-4d38-83ce-4a2cbb4f3843 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:12.269144 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56d404-6ace-4d38-83ce-4a2cbb4f3843 service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:12.269427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56d404-6ace-4d38-83ce-4a2cbb4f3843 service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:12.269698 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56d404-6ace-4d38-83ce-4a2cbb4f3843 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:12.288712 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:12.329039 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56d404-6ace-4d38-83ce-4a2cbb4f3843 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:12.331362 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56d404-6ace-4d38-83ce-4a2cbb4f3843 service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:12.340802 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1383/2769] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:48:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:48:21.164929 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0e64a-cc92-4d65-8dd0-b33ebf690ca6 service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:21.226347 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0e64a-cc92-4d65-8dd0-b33ebf690ca6 service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:21.230865 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01b59c-8e4a-4af9-a089-cef5738a9b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:21.231746 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01b59c-8e4a-4af9-a089-cef5738a9b65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:21.232057 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01b59c-8e4a-4af9-a089-cef5738a9b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:21.232332 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01b59c-8e4a-4af9-a089-cef5738a9b65 admin admin] PATH_INFO: `/v3/users/f005bfb957aa4adfa85f5e09c4c266fc` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:21.237831 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01b59c-8e4a-4af9-a089-cef5738a9b65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f005bfb957aa4adfa85f5e09c4c266fc)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:21.240053 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01b59c-8e4a-4af9-a089-cef5738a9b65 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:21.291684 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-fe01b59c-8e4a-4af9-a089-cef5738a9b65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f005bfb957aa4adfa85f5e09c4c266fc'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:21.299482 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-fe01b59c-8e4a-4af9-a089-cef5738a9b65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f005bfb957aa4adfa85f5e09c4c266fc'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:21.300169 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-fe01b59c-8e4a-4af9-a089-cef5738a9b65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f005bfb957aa4adfa85f5e09c4c266fc'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:21.300890 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-fe01b59c-8e4a-4af9-a089-cef5738a9b65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f005bfb957aa4adfa85f5e09c4c266fc'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:21.311546 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:21.312794 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-fe01b59c-8e4a-4af9-a089-cef5738a9b65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f005bfb957aa4adfa85f5e09c4c266fc'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:21.325644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1387/2770] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:48:21 2022] DELETE /identity/v3/users/f005bfb957aa4adfa85f5e09c4c266fc => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:21.330316 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bed647-6d37-479e-b898-c8da50885b9d None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:21.331019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bed647-6d37-479e-b898-c8da50885b9d None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:21.331019 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bed647-6d37-479e-b898-c8da50885b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:21.349245 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-44bed647-6d37-479e-b898-c8da50885b9d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:21.350475 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-44bed647-6d37-479e-b898-c8da50885b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:21.410238 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1384/2771] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:21.420004 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01b59c-8e4a-4af9-a089-cef5738a9b65 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:21.421412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fe01b59c-8e4a-4af9-a089-cef5738a9b65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:21.478570 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01b59c-8e4a-4af9-a089-cef5738a9b65 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:21.482183 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd48e3a-923e-423e-9245-96609b4e1fb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:21.482812 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd48e3a-923e-423e-9245-96609b4e1fb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:21.483090 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd48e3a-923e-423e-9245-96609b4e1fb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:21.483359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd48e3a-923e-423e-9245-96609b4e1fb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:21.492380 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd48e3a-923e-423e-9245-96609b4e1fb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:21.493793 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd48e3a-923e-423e-9245-96609b4e1fb5 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:21.502988 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1388/2772] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:48:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:48:21.778286 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bed647-6d37-479e-b898-c8da50885b9d None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:21.779747 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-44bed647-6d37-479e-b898-c8da50885b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:21.793252 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bed647-6d37-479e-b898-c8da50885b9d None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:21.798119 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdeda5-948e-49b0-ae6a-fcc2aae9247a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:21.798991 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdeda5-948e-49b0-ae6a-fcc2aae9247a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:21.799277 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdeda5-948e-49b0-ae6a-fcc2aae9247a admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:21.799557 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdeda5-948e-49b0-ae6a-fcc2aae9247a admin admin] PATH_INFO: `/v3/projects/d648b3f0a1034e908d4977aa96586d4d` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:21.801336 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afdeda5-948e-49b0-ae6a-fcc2aae9247a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d648b3f0a1034e908d4977aa96586d4d)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:21.803569 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afdeda5-948e-49b0-ae6a-fcc2aae9247a admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:21.850801 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1afdeda5-948e-49b0-ae6a-fcc2aae9247a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd648b3f0a1034e908d4977aa96586d4d', 'request_id': 'req-1afdeda5-948e-49b0-ae6a-fcc2aae9247a'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:21.856447 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1afdeda5-948e-49b0-ae6a-fcc2aae9247a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd648b3f0a1034e908d4977aa96586d4d', 'request_id': 'req-1afdeda5-948e-49b0-ae6a-fcc2aae9247a'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:21.863933 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1afdeda5-948e-49b0-ae6a-fcc2aae9247a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd648b3f0a1034e908d4977aa96586d4d', 'request_id': 'req-1afdeda5-948e-49b0-ae6a-fcc2aae9247a'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:21.874802 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1afdeda5-948e-49b0-ae6a-fcc2aae9247a admin admin] The token cache is being invalidate because project d648b3f0a1034e908d4977aa96586d4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:48:21.875096 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1afdeda5-948e-49b0-ae6a-fcc2aae9247a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:21.875866 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-1afdeda5-948e-49b0-ae6a-fcc2aae9247a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:21.877329 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1385/2773] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:48:21 2022] DELETE /identity/v3/projects/d648b3f0a1034e908d4977aa96586d4d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:21.882067 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd48e3a-923e-423e-9245-96609b4e1fb5 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:21.891726 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd48e3a-923e-423e-9245-96609b4e1fb5 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:21.946112 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d9f4d-82b5-4445-afa9-f9493f25ad11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:21.946964 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d9f4d-82b5-4445-afa9-f9493f25ad11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:21.947244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d9f4d-82b5-4445-afa9-f9493f25ad11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:21.947517 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d9f4d-82b5-4445-afa9-f9493f25ad11 admin admin] PATH_INFO: `/v3/users/4b02cddd36bd4f79bd4f63d489d5935b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:21.948714 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d9f4d-82b5-4445-afa9-f9493f25ad11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b02cddd36bd4f79bd4f63d489d5935b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:21.950637 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d9f4d-82b5-4445-afa9-f9493f25ad11 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:21.995162 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-741d9f4d-82b5-4445-afa9-f9493f25ad11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b02cddd36bd4f79bd4f63d489d5935b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:22.001925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-741d9f4d-82b5-4445-afa9-f9493f25ad11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b02cddd36bd4f79bd4f63d489d5935b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:22.002569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-741d9f4d-82b5-4445-afa9-f9493f25ad11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b02cddd36bd4f79bd4f63d489d5935b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:22.003165 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-741d9f4d-82b5-4445-afa9-f9493f25ad11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b02cddd36bd4f79bd4f63d489d5935b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:22.016134 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-741d9f4d-82b5-4445-afa9-f9493f25ad11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b02cddd36bd4f79bd4f63d489d5935b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:22.028873 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1389/2774] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:48:21 2022] DELETE /identity/v3/users/4b02cddd36bd4f79bd4f63d489d5935b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:22.301770 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdeda5-948e-49b0-ae6a-fcc2aae9247a admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:22.303248 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-1afdeda5-948e-49b0-ae6a-fcc2aae9247a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:22.316075 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdeda5-948e-49b0-ae6a-fcc2aae9247a admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:22.358074 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:22.374452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d22a1-f13e-4427-a22a-35bff96d1919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:22.375254 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d22a1-f13e-4427-a22a-35bff96d1919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:22.375539 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d22a1-f13e-4427-a22a-35bff96d1919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:22.375851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d22a1-f13e-4427-a22a-35bff96d1919 admin admin] PATH_INFO: `/v3/projects/789c62be7aa04e358983fe3ca2a0a122` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:22.377152 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d22a1-f13e-4427-a22a-35bff96d1919 admin admin] RBAC: Authorizing `identity:delete_project(project_id=789c62be7aa04e358983fe3ca2a0a122)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:22.379170 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d22a1-f13e-4427-a22a-35bff96d1919 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:22.426592 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-c56d22a1-f13e-4427-a22a-35bff96d1919 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '789c62be7aa04e358983fe3ca2a0a122', 'request_id': 'req-c56d22a1-f13e-4427-a22a-35bff96d1919'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:22.432238 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-c56d22a1-f13e-4427-a22a-35bff96d1919 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '789c62be7aa04e358983fe3ca2a0a122', 'request_id': 'req-c56d22a1-f13e-4427-a22a-35bff96d1919'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:22.439181 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-c56d22a1-f13e-4427-a22a-35bff96d1919 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '789c62be7aa04e358983fe3ca2a0a122', 'request_id': 'req-c56d22a1-f13e-4427-a22a-35bff96d1919'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:22.449983 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-c56d22a1-f13e-4427-a22a-35bff96d1919 admin admin] The token cache is being invalidate because project 789c62be7aa04e358983fe3ca2a0a122 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:48:22.450264 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-c56d22a1-f13e-4427-a22a-35bff96d1919 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:22.450897 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-c56d22a1-f13e-4427-a22a-35bff96d1919 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:22.452454 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1386/2775] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:48:22 2022] DELETE /identity/v3/projects/789c62be7aa04e358983fe3ca2a0a122 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:22.457538 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ee971-af6e-428b-a93b-d13e99ae0b9e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:22.458251 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ee971-af6e-428b-a93b-d13e99ae0b9e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:22.458428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ee971-af6e-428b-a93b-d13e99ae0b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:22.476243 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-d34ee971-af6e-428b-a93b-d13e99ae0b9e None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:22.477468 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d34ee971-af6e-428b-a93b-d13e99ae0b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:22.482700 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:22.539628 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1390/2776] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:22.548867 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecbdcee-51b5-4071-b59c-4a293131e4cd None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:22.549156 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecbdcee-51b5-4071-b59c-4a293131e4cd None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:22.549435 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecbdcee-51b5-4071-b59c-4a293131e4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:22.567452 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-cecbdcee-51b5-4071-b59c-4a293131e4cd None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:22.568678 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-cecbdcee-51b5-4071-b59c-4a293131e4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:22.576446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1387/2777] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:22.580802 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ee971-af6e-428b-a93b-d13e99ae0b9e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:22.591084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ee971-af6e-428b-a93b-d13e99ae0b9e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:22.595707 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5023832-857e-422a-93a9-870350349b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:22.596390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5023832-857e-422a-93a9-870350349b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:22.596683 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5023832-857e-422a-93a9-870350349b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:22.596961 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5023832-857e-422a-93a9-870350349b43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:22.598348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5023832-857e-422a-93a9-870350349b43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:48:22.598786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5023832-857e-422a-93a9-870350349b43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:22.600392 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5023832-857e-422a-93a9-870350349b43 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:22.611732 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1391/2778] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:48:22.616013 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecbdcee-51b5-4071-b59c-4a293131e4cd None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:22.625083 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecbdcee-51b5-4071-b59c-4a293131e4cd None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:22.629659 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d483af-01e3-4d1d-b917-944ef3d542a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:22.630291 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d483af-01e3-4d1d-b917-944ef3d542a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:22.630570 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d483af-01e3-4d1d-b917-944ef3d542a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:22.630842 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d483af-01e3-4d1d-b917-944ef3d542a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:22.631523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d483af-01e3-4d1d-b917-944ef3d542a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:22.633415 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d483af-01e3-4d1d-b917-944ef3d542a6 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:22.657977 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1388/2779] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:48:22 2022] POST /identity/v3/projects => generated 389 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:48:22.662694 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5023832-857e-422a-93a9-870350349b43 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:22.672581 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5023832-857e-422a-93a9-870350349b43 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:22.726848 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2f48f-e962-4239-b7f6-aee78cd37994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:22.727504 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2f48f-e962-4239-b7f6-aee78cd37994 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:22.727803 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2f48f-e962-4239-b7f6-aee78cd37994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:22.728088 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2f48f-e962-4239-b7f6-aee78cd37994 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:22.728773 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2f48f-e962-4239-b7f6-aee78cd37994 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:22.730656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2f48f-e962-4239-b7f6-aee78cd37994 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:22.750897 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1392/2780] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:48:22 2022] POST /identity/v3/users => generated 295 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:48:22.755185 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d483af-01e3-4d1d-b917-944ef3d542a6 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:22.765715 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d483af-01e3-4d1d-b917-944ef3d542a6 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:22.771010 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296671d-a3b0-43f1-b1de-a07aba7c107c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:22.771666 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296671d-a3b0-43f1-b1de-a07aba7c107c admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:22.771985 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296671d-a3b0-43f1-b1de-a07aba7c107c admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:22.772268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296671d-a3b0-43f1-b1de-a07aba7c107c admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:22.773018 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296671d-a3b0-43f1-b1de-a07aba7c107c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:22.774615 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296671d-a3b0-43f1-b1de-a07aba7c107c admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:22.784861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1389/2781] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:48:22.789430 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2f48f-e962-4239-b7f6-aee78cd37994 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:22.799486 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2f48f-e962-4239-b7f6-aee78cd37994 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:22.804643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16daa22-68cb-4ce2-b80d-e845ff70c753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:22.805296 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16daa22-68cb-4ce2-b80d-e845ff70c753 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:22.805579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16daa22-68cb-4ce2-b80d-e845ff70c753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:22.805856 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16daa22-68cb-4ce2-b80d-e845ff70c753 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:22.806573 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16daa22-68cb-4ce2-b80d-e845ff70c753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:22.808202 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16daa22-68cb-4ce2-b80d-e845ff70c753 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:22.817987 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1393/2782] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:48:22.821889 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296671d-a3b0-43f1-b1de-a07aba7c107c admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:22.831714 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296671d-a3b0-43f1-b1de-a07aba7c107c admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:22.836920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0d7e0-4334-48b5-ad19-d08a1591c868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:22.837795 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0d7e0-4334-48b5-ad19-d08a1591c868 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:22.838080 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0d7e0-4334-48b5-ad19-d08a1591c868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:22.838356 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0d7e0-4334-48b5-ad19-d08a1591c868 admin admin] PATH_INFO: `/v3/projects/40e6e5b9e1c24452942a2edc10d6db28/users/22db60fa5a72438d92978353b7420034/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:22.853135 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0d7e0-4334-48b5-ad19-d08a1591c868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40e6e5b9e1c24452942a2edc10d6db28, user_id=22db60fa5a72438d92978353b7420034, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:22.855747 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0d7e0-4334-48b5-ad19-d08a1591c868 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:22.868324 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1390/2783] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:48:22 2022] PUT /identity/v3/projects/40e6e5b9e1c24452942a2edc10d6db28/users/22db60fa5a72438d92978353b7420034/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:22.880359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6a536-ec2e-4389-8eff-047b8c8471dd None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:22.880588 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6a536-ec2e-4389-8eff-047b8c8471dd None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:22.880867 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6a536-ec2e-4389-8eff-047b8c8471dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:22.907088 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-81c6a536-ec2e-4389-8eff-047b8c8471dd None None] MFA Rules not processed for user `22db60fa5a72438d92978353b7420034`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:22.908335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-81c6a536-ec2e-4389-8eff-047b8c8471dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:22.970260 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1394/2784] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:22.975450 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0d7e0-4334-48b5-ad19-d08a1591c868 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:23.038679 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0d7e0-4334-48b5-ad19-d08a1591c868 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:23.043358 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d42042-fd3d-4762-9640-098a2aa4ecc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:23.044042 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d42042-fd3d-4762-9640-098a2aa4ecc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:23.044330 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d42042-fd3d-4762-9640-098a2aa4ecc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:23.044609 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d42042-fd3d-4762-9640-098a2aa4ecc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:23.045326 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d42042-fd3d-4762-9640-098a2aa4ecc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:23.047217 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d42042-fd3d-4762-9640-098a2aa4ecc0 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:23.061522 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:23.074851 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1391/2785] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:48:22 2022] POST /identity/v3/projects => generated 391 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:48:23.078822 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6a536-ec2e-4389-8eff-047b8c8471dd None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:23.089636 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6a536-ec2e-4389-8eff-047b8c8471dd None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:23.093189 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:23.147807 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51907e7b-48c5-406c-a66c-3d19282496a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:23.148399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51907e7b-48c5-406c-a66c-3d19282496a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:23.148688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51907e7b-48c5-406c-a66c-3d19282496a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:23.148964 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51907e7b-48c5-406c-a66c-3d19282496a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:23.149659 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51907e7b-48c5-406c-a66c-3d19282496a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:23.151561 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51907e7b-48c5-406c-a66c-3d19282496a8 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:23.170813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1395/2786] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:48:23 2022] POST /identity/v3/users => generated 302 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:48:23.174805 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d42042-fd3d-4762-9640-098a2aa4ecc0 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:23.185799 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d42042-fd3d-4762-9640-098a2aa4ecc0 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:23.190439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8f3c5-9c1b-4971-98a5-b83668b47160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:23.191090 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8f3c5-9c1b-4971-98a5-b83668b47160 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:23.191382 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8f3c5-9c1b-4971-98a5-b83668b47160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:23.191732 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8f3c5-9c1b-4971-98a5-b83668b47160 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:23.192496 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f8f3c5-9c1b-4971-98a5-b83668b47160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:23.194067 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f8f3c5-9c1b-4971-98a5-b83668b47160 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:23.204650 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1392/2787] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:48:23.208912 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51907e7b-48c5-406c-a66c-3d19282496a8 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:23.218784 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51907e7b-48c5-406c-a66c-3d19282496a8 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:23.224043 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7d694-d098-44f6-b593-a09fdaf3b6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:23.224925 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7d694-d098-44f6-b593-a09fdaf3b6a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:23.225213 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7d694-d098-44f6-b593-a09fdaf3b6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:23.225488 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7d694-d098-44f6-b593-a09fdaf3b6a8 admin admin] PATH_INFO: `/v3/projects/0bd4f81ecb7d4166810b3464eb30cb37/users/7dabe937c9b44afba1ed51e17662a43a/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:23.240673 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7d694-d098-44f6-b593-a09fdaf3b6a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bd4f81ecb7d4166810b3464eb30cb37, user_id=7dabe937c9b44afba1ed51e17662a43a, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:23.243304 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7d694-d098-44f6-b593-a09fdaf3b6a8 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:23.256556 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1396/2788] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:48:23 2022] PUT /identity/v3/projects/0bd4f81ecb7d4166810b3464eb30cb37/users/7dabe937c9b44afba1ed51e17662a43a/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:23.260723 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8f3c5-9c1b-4971-98a5-b83668b47160 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:23.271506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8f3c5-9c1b-4971-98a5-b83668b47160 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:23.328241 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8b779-39cf-4ca0-a577-cf26583fa87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:23.329112 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8b779-39cf-4ca0-a577-cf26583fa87e admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:23.329253 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8b779-39cf-4ca0-a577-cf26583fa87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:23.329466 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8b779-39cf-4ca0-a577-cf26583fa87e admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:23.330411 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8b779-39cf-4ca0-a577-cf26583fa87e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:23.332515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8b779-39cf-4ca0-a577-cf26583fa87e admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:23.342613 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1393/2789] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:48:23.346747 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7d694-d098-44f6-b593-a09fdaf3b6a8 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:23.356824 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7d694-d098-44f6-b593-a09fdaf3b6a8 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:23.361428 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8fc69-520f-4305-a60d-6906e732cc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:23.362297 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8fc69-520f-4305-a60d-6906e732cc2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:23.362579 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8fc69-520f-4305-a60d-6906e732cc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:23.362855 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8fc69-520f-4305-a60d-6906e732cc2b admin admin] PATH_INFO: `/v3/projects/0bd4f81ecb7d4166810b3464eb30cb37/users/7dabe937c9b44afba1ed51e17662a43a/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:23.365535 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8fc69-520f-4305-a60d-6906e732cc2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bd4f81ecb7d4166810b3464eb30cb37, user_id=7dabe937c9b44afba1ed51e17662a43a, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:23.368187 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8fc69-520f-4305-a60d-6906e732cc2b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:23.378269 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1397/2790] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:48:23 2022] PUT /identity/v3/projects/0bd4f81ecb7d4166810b3464eb30cb37/users/7dabe937c9b44afba1ed51e17662a43a/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:23.393051 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954121b4-90f7-4652-8b00-3c7678fb7a67 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:23.393476 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954121b4-90f7-4652-8b00-3c7678fb7a67 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:23.393624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954121b4-90f7-4652-8b00-3c7678fb7a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:23.419809 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-954121b4-90f7-4652-8b00-3c7678fb7a67 None None] MFA Rules not processed for user `7dabe937c9b44afba1ed51e17662a43a`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:23.420972 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-954121b4-90f7-4652-8b00-3c7678fb7a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:23.487725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1394/2791] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:23.499199 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8fc69-520f-4305-a60d-6906e732cc2b admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:23.500988 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b0a8fc69-520f-4305-a60d-6906e732cc2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:23.557861 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8fc69-520f-4305-a60d-6906e732cc2b admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:23.561564 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632266d-d3c0-4b66-a3de-5c99dcd55e06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:23.562207 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632266d-d3c0-4b66-a3de-5c99dcd55e06 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:23.562491 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632266d-d3c0-4b66-a3de-5c99dcd55e06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:23.562765 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632266d-d3c0-4b66-a3de-5c99dcd55e06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:23.573286 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632266d-d3c0-4b66-a3de-5c99dcd55e06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:23.574722 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632266d-d3c0-4b66-a3de-5c99dcd55e06 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:23.584388 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1398/2792] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:48:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:48:23.630979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954121b4-90f7-4652-8b00-3c7678fb7a67 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:23.632541 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-954121b4-90f7-4652-8b00-3c7678fb7a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:23.645782 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954121b4-90f7-4652-8b00-3c7678fb7a67 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:23.649369 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36e7ae-7120-4700-8344-3b12d664422a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:23.650002 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36e7ae-7120-4700-8344-3b12d664422a service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:23.650283 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36e7ae-7120-4700-8344-3b12d664422a service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:23.650561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36e7ae-7120-4700-8344-3b12d664422a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:23.709670 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36e7ae-7120-4700-8344-3b12d664422a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:23.711146 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36e7ae-7120-4700-8344-3b12d664422a service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:23.720370 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1395/2793] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:48:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:48:26.938168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632266d-d3c0-4b66-a3de-5c99dcd55e06 service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:26.999051 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632266d-d3c0-4b66-a3de-5c99dcd55e06 service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:27.003747 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d320f4b-137d-408b-b591-d3f9b395d0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:27.005050 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d320f4b-137d-408b-b591-d3f9b395d0d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:27.005341 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d320f4b-137d-408b-b591-d3f9b395d0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:27.005619 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d320f4b-137d-408b-b591-d3f9b395d0d5 admin admin] PATH_INFO: `/v3/users/22db60fa5a72438d92978353b7420034` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:27.006813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d320f4b-137d-408b-b591-d3f9b395d0d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22db60fa5a72438d92978353b7420034)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:27.008777 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d320f4b-137d-408b-b591-d3f9b395d0d5 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:27.011114 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:27.055405 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-9d320f4b-137d-408b-b591-d3f9b395d0d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22db60fa5a72438d92978353b7420034'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:27.062832 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-9d320f4b-137d-408b-b591-d3f9b395d0d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22db60fa5a72438d92978353b7420034'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:27.063495 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-9d320f4b-137d-408b-b591-d3f9b395d0d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22db60fa5a72438d92978353b7420034'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:27.064110 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-9d320f4b-137d-408b-b591-d3f9b395d0d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22db60fa5a72438d92978353b7420034'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:27.076803 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-9d320f4b-137d-408b-b591-d3f9b395d0d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22db60fa5a72438d92978353b7420034'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:27.089249 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1399/2794] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:48:26 2022] DELETE /identity/v3/users/22db60fa5a72438d92978353b7420034 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:27.093706 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f81e90-df8e-4a3b-ace9-266b78a6e9b5 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:27.094187 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f81e90-df8e-4a3b-ace9-266b78a6e9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:27.094301 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f81e90-df8e-4a3b-ace9-266b78a6e9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:27.100784 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=100 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:27.111236 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-e2f81e90-df8e-4a3b-ace9-266b78a6e9b5 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:27.112454 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-e2f81e90-df8e-4a3b-ace9-266b78a6e9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:27.171988 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1396/2795] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:27.182103 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d320f4b-137d-408b-b591-d3f9b395d0d5 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:27.183503 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9d320f4b-137d-408b-b591-d3f9b395d0d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:27.241145 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d320f4b-137d-408b-b591-d3f9b395d0d5 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:27.244892 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b63dc-8b5c-4018-a8fd-cea4e8e63901 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:27.245537 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b63dc-8b5c-4018-a8fd-cea4e8e63901 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:27.245821 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b63dc-8b5c-4018-a8fd-cea4e8e63901 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:27.246101 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b63dc-8b5c-4018-a8fd-cea4e8e63901 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:27.255918 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b63dc-8b5c-4018-a8fd-cea4e8e63901 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:27.257381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b63dc-8b5c-4018-a8fd-cea4e8e63901 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:27.266957 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1400/2796] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:48:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:48:27.547249 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f81e90-df8e-4a3b-ace9-266b78a6e9b5 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:27.548732 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-e2f81e90-df8e-4a3b-ace9-266b78a6e9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:27.563970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f81e90-df8e-4a3b-ace9-266b78a6e9b5 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:27.569663 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d213e10-14d8-4137-8b13-491e8444f3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:27.570487 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d213e10-14d8-4137-8b13-491e8444f3b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:27.570770 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d213e10-14d8-4137-8b13-491e8444f3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:27.571046 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d213e10-14d8-4137-8b13-491e8444f3b2 admin admin] PATH_INFO: `/v3/projects/40e6e5b9e1c24452942a2edc10d6db28` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:27.572447 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d213e10-14d8-4137-8b13-491e8444f3b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40e6e5b9e1c24452942a2edc10d6db28)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:27.574385 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d213e10-14d8-4137-8b13-491e8444f3b2 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:27.620669 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-4d213e10-14d8-4137-8b13-491e8444f3b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40e6e5b9e1c24452942a2edc10d6db28', 'request_id': 'req-4d213e10-14d8-4137-8b13-491e8444f3b2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:27.625947 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-4d213e10-14d8-4137-8b13-491e8444f3b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40e6e5b9e1c24452942a2edc10d6db28', 'request_id': 'req-4d213e10-14d8-4137-8b13-491e8444f3b2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:27.633306 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-4d213e10-14d8-4137-8b13-491e8444f3b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40e6e5b9e1c24452942a2edc10d6db28', 'request_id': 'req-4d213e10-14d8-4137-8b13-491e8444f3b2'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:27.644680 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-4d213e10-14d8-4137-8b13-491e8444f3b2 admin admin] The token cache is being invalidate because project 40e6e5b9e1c24452942a2edc10d6db28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:48:27.644998 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-4d213e10-14d8-4137-8b13-491e8444f3b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:27.645624 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-4d213e10-14d8-4137-8b13-491e8444f3b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:27.647172 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1397/2797] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:48:27 2022] DELETE /identity/v3/projects/40e6e5b9e1c24452942a2edc10d6db28 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:27.651236 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b63dc-8b5c-4018-a8fd-cea4e8e63901 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:27.661389 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b63dc-8b5c-4018-a8fd-cea4e8e63901 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:27.716994 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70270fed-8fc8-4fad-b9d0-99b4bb84573e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:27.717861 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70270fed-8fc8-4fad-b9d0-99b4bb84573e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:27.718139 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70270fed-8fc8-4fad-b9d0-99b4bb84573e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:27.718411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70270fed-8fc8-4fad-b9d0-99b4bb84573e admin admin] PATH_INFO: `/v3/users/7dabe937c9b44afba1ed51e17662a43a` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:27.719597 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70270fed-8fc8-4fad-b9d0-99b4bb84573e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dabe937c9b44afba1ed51e17662a43a)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:27.721564 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70270fed-8fc8-4fad-b9d0-99b4bb84573e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:27.749031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:27.769106 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-70270fed-8fc8-4fad-b9d0-99b4bb84573e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dabe937c9b44afba1ed51e17662a43a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:27.776581 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-70270fed-8fc8-4fad-b9d0-99b4bb84573e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dabe937c9b44afba1ed51e17662a43a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:27.777255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-70270fed-8fc8-4fad-b9d0-99b4bb84573e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dabe937c9b44afba1ed51e17662a43a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:27.777881 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-70270fed-8fc8-4fad-b9d0-99b4bb84573e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dabe937c9b44afba1ed51e17662a43a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:27.790092 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-70270fed-8fc8-4fad-b9d0-99b4bb84573e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dabe937c9b44afba1ed51e17662a43a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:27.803016 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1401/2798] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:48:27 2022] DELETE /identity/v3/users/7dabe937c9b44afba1ed51e17662a43a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:28.072151 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d213e10-14d8-4137-8b13-491e8444f3b2 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:28.073610 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-4d213e10-14d8-4137-8b13-491e8444f3b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:28.087092 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d213e10-14d8-4137-8b13-491e8444f3b2 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:28.143041 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f16937-0bd3-43ed-8067-a4227bfa3516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:28.143932 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f16937-0bd3-43ed-8067-a4227bfa3516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:28.144219 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f16937-0bd3-43ed-8067-a4227bfa3516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:28.144497 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f16937-0bd3-43ed-8067-a4227bfa3516 admin admin] PATH_INFO: `/v3/projects/0bd4f81ecb7d4166810b3464eb30cb37` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:28.146228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f16937-0bd3-43ed-8067-a4227bfa3516 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bd4f81ecb7d4166810b3464eb30cb37)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:28.148204 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f16937-0bd3-43ed-8067-a4227bfa3516 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:28.180135 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:28.194133 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-63f16937-0bd3-43ed-8067-a4227bfa3516 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bd4f81ecb7d4166810b3464eb30cb37', 'request_id': 'req-63f16937-0bd3-43ed-8067-a4227bfa3516'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:28.199530 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-63f16937-0bd3-43ed-8067-a4227bfa3516 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bd4f81ecb7d4166810b3464eb30cb37', 'request_id': 'req-63f16937-0bd3-43ed-8067-a4227bfa3516'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:28.205957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-63f16937-0bd3-43ed-8067-a4227bfa3516 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bd4f81ecb7d4166810b3464eb30cb37', 'request_id': 'req-63f16937-0bd3-43ed-8067-a4227bfa3516'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:28.217728 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-63f16937-0bd3-43ed-8067-a4227bfa3516 admin admin] The token cache is being invalidate because project 0bd4f81ecb7d4166810b3464eb30cb37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:48:28.218018 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-63f16937-0bd3-43ed-8067-a4227bfa3516 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:28.218651 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-63f16937-0bd3-43ed-8067-a4227bfa3516 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:28.220258 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1398/2799] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:48:28 2022] DELETE /identity/v3/projects/0bd4f81ecb7d4166810b3464eb30cb37 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:28.225224 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93df5bb-1008-41d4-be06-07ad4706f415 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:28.225748 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93df5bb-1008-41d4-be06-07ad4706f415 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:28.225977 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93df5bb-1008-41d4-be06-07ad4706f415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:28.243066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c93df5bb-1008-41d4-be06-07ad4706f415 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:28.244363 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c93df5bb-1008-41d4-be06-07ad4706f415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:28.302042 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1402/2800] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:28.310683 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a7b89-7511-43d6-8be5-a71cd202fc9d None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:28.310978 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a7b89-7511-43d6-8be5-a71cd202fc9d None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:28.311260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a7b89-7511-43d6-8be5-a71cd202fc9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:28.330088 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-914a7b89-7511-43d6-8be5-a71cd202fc9d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:28.331460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-914a7b89-7511-43d6-8be5-a71cd202fc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:28.340173 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1399/2801] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 10:48:28.344263 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93df5bb-1008-41d4-be06-07ad4706f415 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:28.354278 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93df5bb-1008-41d4-be06-07ad4706f415 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:28.359131 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bc7d2-c324-492f-a69c-0b9f7001421c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:28.359811 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bc7d2-c324-492f-a69c-0b9f7001421c admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:28.360096 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bc7d2-c324-492f-a69c-0b9f7001421c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:28.360405 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bc7d2-c324-492f-a69c-0b9f7001421c admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:28.361160 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20bc7d2-c324-492f-a69c-0b9f7001421c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:48:28.361468 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20bc7d2-c324-492f-a69c-0b9f7001421c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:28.363064 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20bc7d2-c324-492f-a69c-0b9f7001421c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:28.374635 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1403/2802] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:48:28.379488 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a7b89-7511-43d6-8be5-a71cd202fc9d None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:28.388937 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a7b89-7511-43d6-8be5-a71cd202fc9d None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:28.394034 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc1d5b-7572-41a4-8674-63adf52460f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:28.395317 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc1d5b-7572-41a4-8674-63adf52460f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:28.395598 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc1d5b-7572-41a4-8674-63adf52460f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:28.395908 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc1d5b-7572-41a4-8674-63adf52460f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:28.396588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc1d5b-7572-41a4-8674-63adf52460f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:28.398562 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc1d5b-7572-41a4-8674-63adf52460f7 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:28.423726 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1400/2803] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:48:28 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:48:28.427958 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bc7d2-c324-492f-a69c-0b9f7001421c admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:28.437796 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bc7d2-c324-492f-a69c-0b9f7001421c admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:28.493238 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82410525-669b-4fbe-96b5-ac87b1aef767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:28.493897 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82410525-669b-4fbe-96b5-ac87b1aef767 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:28.494179 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82410525-669b-4fbe-96b5-ac87b1aef767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:28.494453 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82410525-669b-4fbe-96b5-ac87b1aef767 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:28.495129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82410525-669b-4fbe-96b5-ac87b1aef767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:28.497068 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82410525-669b-4fbe-96b5-ac87b1aef767 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:28.517046 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1404/2804] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:48:28 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:48:28.521751 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc1d5b-7572-41a4-8674-63adf52460f7 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:28.531754 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc1d5b-7572-41a4-8674-63adf52460f7 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:28.536286 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23e234-0690-495e-8baf-67a0ba6a8fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:28.536933 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23e234-0690-495e-8baf-67a0ba6a8fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:28.537212 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23e234-0690-495e-8baf-67a0ba6a8fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:28.537482 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23e234-0690-495e-8baf-67a0ba6a8fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:28.538190 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa23e234-0690-495e-8baf-67a0ba6a8fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:28.539760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa23e234-0690-495e-8baf-67a0ba6a8fb8 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:28.550276 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1401/2805] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:48:28.554792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82410525-669b-4fbe-96b5-ac87b1aef767 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:28.564726 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82410525-669b-4fbe-96b5-ac87b1aef767 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:28.569968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e1dfd-81f7-46e3-a09d-567503013685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:28.570640 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e1dfd-81f7-46e3-a09d-567503013685 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:28.570926 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e1dfd-81f7-46e3-a09d-567503013685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:28.571204 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e1dfd-81f7-46e3-a09d-567503013685 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:28.571999 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e1dfd-81f7-46e3-a09d-567503013685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:28.573581 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e1dfd-81f7-46e3-a09d-567503013685 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:28.583376 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1405/2806] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:48:28.587511 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23e234-0690-495e-8baf-67a0ba6a8fb8 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:28.597336 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23e234-0690-495e-8baf-67a0ba6a8fb8 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:28.601874 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843eb0b4-6591-4920-8045-f83ecac60cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:28.602737 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843eb0b4-6591-4920-8045-f83ecac60cda admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:28.603018 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843eb0b4-6591-4920-8045-f83ecac60cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:28.603284 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843eb0b4-6591-4920-8045-f83ecac60cda admin admin] PATH_INFO: `/v3/projects/d4c81fed3e1143edbb156e058f33b6ae/users/91d036196c1e4e07a4b13a334bbff646/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:28.617930 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843eb0b4-6591-4920-8045-f83ecac60cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4c81fed3e1143edbb156e058f33b6ae, user_id=91d036196c1e4e07a4b13a334bbff646, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:28.620572 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843eb0b4-6591-4920-8045-f83ecac60cda admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:28.633543 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1402/2807] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:48:28 2022] PUT /identity/v3/projects/d4c81fed3e1143edbb156e058f33b6ae/users/91d036196c1e4e07a4b13a334bbff646/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:48:28.693302 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de4e2e-e669-4f1f-ac7c-8526f483c839 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:28.693590 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de4e2e-e669-4f1f-ac7c-8526f483c839 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:28.693869 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de4e2e-e669-4f1f-ac7c-8526f483c839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:28.712234 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:28.722545 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-31de4e2e-e669-4f1f-ac7c-8526f483c839 None None] MFA Rules not processed for user `91d036196c1e4e07a4b13a334bbff646`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:28.724580 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-31de4e2e-e669-4f1f-ac7c-8526f483c839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:28.789276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1406/2808] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:28.793748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843eb0b4-6591-4920-8045-f83ecac60cda admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:28.856356 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843eb0b4-6591-4920-8045-f83ecac60cda admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:28.860943 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89647e-457d-4ed1-b0e0-1e1244441cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:28.861587 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89647e-457d-4ed1-b0e0-1e1244441cda admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:28.861866 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89647e-457d-4ed1-b0e0-1e1244441cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:28.862138 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89647e-457d-4ed1-b0e0-1e1244441cda admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:28.862826 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc89647e-457d-4ed1-b0e0-1e1244441cda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:28.865561 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc89647e-457d-4ed1-b0e0-1e1244441cda admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:28.890594 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1403/2809] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:48:28 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:48:28.894665 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de4e2e-e669-4f1f-ac7c-8526f483c839 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:28.905198 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de4e2e-e669-4f1f-ac7c-8526f483c839 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:28.964375 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a4d78-a62e-4d2d-a61f-03f1f84f1628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:28.964968 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a4d78-a62e-4d2d-a61f-03f1f84f1628 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:28.965252 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a4d78-a62e-4d2d-a61f-03f1f84f1628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:28.965526 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a4d78-a62e-4d2d-a61f-03f1f84f1628 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:28.966223 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a4d78-a62e-4d2d-a61f-03f1f84f1628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:28.968141 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a4d78-a62e-4d2d-a61f-03f1f84f1628 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:28.987962 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1407/2810] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:48:28 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:48:28.992598 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89647e-457d-4ed1-b0e0-1e1244441cda admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:29.002779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89647e-457d-4ed1-b0e0-1e1244441cda admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:29.008079 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf84767-81d5-4ac5-a2ef-c4507252aade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:29.008748 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf84767-81d5-4ac5-a2ef-c4507252aade admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:29.009036 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf84767-81d5-4ac5-a2ef-c4507252aade admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:29.009314 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf84767-81d5-4ac5-a2ef-c4507252aade admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:29.010040 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf84767-81d5-4ac5-a2ef-c4507252aade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:29.011628 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf84767-81d5-4ac5-a2ef-c4507252aade admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:29.022946 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1404/2811] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:48:29.026908 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a4d78-a62e-4d2d-a61f-03f1f84f1628 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:29.037177 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a4d78-a62e-4d2d-a61f-03f1f84f1628 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:29.041675 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3730d-bf06-45ef-8df6-45f1522a3991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:29.042320 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3730d-bf06-45ef-8df6-45f1522a3991 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:29.042600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3730d-bf06-45ef-8df6-45f1522a3991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:29.043104 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3730d-bf06-45ef-8df6-45f1522a3991 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:29.043848 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3730d-bf06-45ef-8df6-45f1522a3991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:29.045412 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3730d-bf06-45ef-8df6-45f1522a3991 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:29.055865 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1408/2812] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:48:29.059768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf84767-81d5-4ac5-a2ef-c4507252aade admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:29.069688 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf84767-81d5-4ac5-a2ef-c4507252aade admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:29.074312 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cccacec-89bb-498e-b2bd-00e550601d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:29.075188 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cccacec-89bb-498e-b2bd-00e550601d39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:29.075513 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cccacec-89bb-498e-b2bd-00e550601d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:29.075899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cccacec-89bb-498e-b2bd-00e550601d39 admin admin] PATH_INFO: `/v3/projects/1f4801c2c9104201919853c36aab938e/users/a31d151359d648899720bfc724c9695f/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:29.090308 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cccacec-89bb-498e-b2bd-00e550601d39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f4801c2c9104201919853c36aab938e, user_id=a31d151359d648899720bfc724c9695f, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:29.092969 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cccacec-89bb-498e-b2bd-00e550601d39 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:29.106374 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1405/2813] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:48:29 2022] PUT /identity/v3/projects/1f4801c2c9104201919853c36aab938e/users/a31d151359d648899720bfc724c9695f/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:29.117292 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe0ea6-d55b-4942-9f06-052b2abef3b1 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:29.117616 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe0ea6-d55b-4942-9f06-052b2abef3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:29.117898 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe0ea6-d55b-4942-9f06-052b2abef3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:29.144395 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-8abe0ea6-d55b-4942-9f06-052b2abef3b1 None None] MFA Rules not processed for user `a31d151359d648899720bfc724c9695f`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:29.145615 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-8abe0ea6-d55b-4942-9f06-052b2abef3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:29.208559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1409/2814] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:29.213074 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cccacec-89bb-498e-b2bd-00e550601d39 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:29.218719 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:29.293799 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cccacec-89bb-498e-b2bd-00e550601d39 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:29.298606 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36a92a-96ba-406e-9e24-04c5b42e1a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:29.299275 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36a92a-96ba-406e-9e24-04c5b42e1a32 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:29.299559 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36a92a-96ba-406e-9e24-04c5b42e1a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:29.299897 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36a92a-96ba-406e-9e24-04c5b42e1a32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:29.300649 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36a92a-96ba-406e-9e24-04c5b42e1a32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:29.302839 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36a92a-96ba-406e-9e24-04c5b42e1a32 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:29.331768 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1406/2815] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:48:29 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 10:48:29.333556 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe0ea6-d55b-4942-9f06-052b2abef3b1 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:29.344307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe0ea6-d55b-4942-9f06-052b2abef3b1 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:29.348310 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:29.402656 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91d253-13ac-4120-8680-843dec0df1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:29.403323 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91d253-13ac-4120-8680-843dec0df1e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:29.403603 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91d253-13ac-4120-8680-843dec0df1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:29.403907 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91d253-13ac-4120-8680-843dec0df1e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:29.404600 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a91d253-13ac-4120-8680-843dec0df1e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:29.406494 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a91d253-13ac-4120-8680-843dec0df1e6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:29.426071 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1410/2816] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:48:29 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:48:29.430152 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36a92a-96ba-406e-9e24-04c5b42e1a32 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:29.440506 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36a92a-96ba-406e-9e24-04c5b42e1a32 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:29.445270 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f3657-a243-4627-a71f-855f4031e869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:29.445922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f3657-a243-4627-a71f-855f4031e869 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:29.446209 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f3657-a243-4627-a71f-855f4031e869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:29.446484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f3657-a243-4627-a71f-855f4031e869 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:29.447199 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f3657-a243-4627-a71f-855f4031e869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:29.448833 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f3657-a243-4627-a71f-855f4031e869 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:29.459723 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1407/2817] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:48:29.463958 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91d253-13ac-4120-8680-843dec0df1e6 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:29.474748 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91d253-13ac-4120-8680-843dec0df1e6 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:29.479297 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8add002-8d93-41fc-be6f-a857ea010ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:29.480212 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8add002-8d93-41fc-be6f-a857ea010ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:29.480499 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8add002-8d93-41fc-be6f-a857ea010ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:29.480772 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8add002-8d93-41fc-be6f-a857ea010ba8 admin admin] PATH_INFO: `/v3/projects/6bf903d8a2694c3d892491002b1d7df6/users/87243418818342cc84a4967b6cfcf3d9/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:29.495450 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8add002-8d93-41fc-be6f-a857ea010ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bf903d8a2694c3d892491002b1d7df6, user_id=87243418818342cc84a4967b6cfcf3d9, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:29.498139 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8add002-8d93-41fc-be6f-a857ea010ba8 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:29.512352 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1411/2818] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:48:29 2022] PUT /identity/v3/projects/6bf903d8a2694c3d892491002b1d7df6/users/87243418818342cc84a4967b6cfcf3d9/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:48:29.516523 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f3657-a243-4627-a71f-855f4031e869 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:29.527590 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f3657-a243-4627-a71f-855f4031e869 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:29.605365 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7d796-d396-4140-b4ef-89e36e595298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:29.610089 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7d796-d396-4140-b4ef-89e36e595298 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:29.610406 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7d796-d396-4140-b4ef-89e36e595298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:29.610698 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7d796-d396-4140-b4ef-89e36e595298 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:29.611543 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be7d796-d396-4140-b4ef-89e36e595298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:29.613882 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be7d796-d396-4140-b4ef-89e36e595298 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:29.627511 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1408/2819] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:48:29.632378 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8add002-8d93-41fc-be6f-a857ea010ba8 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:29.643024 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8add002-8d93-41fc-be6f-a857ea010ba8 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:29.648438 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f731c-57da-4b4d-b9a3-e2a214f1c02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:29.649312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f731c-57da-4b4d-b9a3-e2a214f1c02e admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:29.649594 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f731c-57da-4b4d-b9a3-e2a214f1c02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:29.649869 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f731c-57da-4b4d-b9a3-e2a214f1c02e admin admin] PATH_INFO: `/v3/projects/6bf903d8a2694c3d892491002b1d7df6/users/87243418818342cc84a4967b6cfcf3d9/roles/0f929204eb7b4ea8859338ed2d3ff75b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:29.651875 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f731c-57da-4b4d-b9a3-e2a214f1c02e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bf903d8a2694c3d892491002b1d7df6, user_id=87243418818342cc84a4967b6cfcf3d9, role_id=0f929204eb7b4ea8859338ed2d3ff75b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:29.654490 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f731c-57da-4b4d-b9a3-e2a214f1c02e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:29.664657 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1412/2820] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:48:29 2022] PUT /identity/v3/projects/6bf903d8a2694c3d892491002b1d7df6/users/87243418818342cc84a4967b6cfcf3d9/roles/0f929204eb7b4ea8859338ed2d3ff75b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:29.675918 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4acb46-c1b2-4356-b94a-11b128d10865 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:29.676322 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4acb46-c1b2-4356-b94a-11b128d10865 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:29.676640 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4acb46-c1b2-4356-b94a-11b128d10865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:29.703500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-ec4acb46-c1b2-4356-b94a-11b128d10865 None None] MFA Rules not processed for user `87243418818342cc84a4967b6cfcf3d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:29.704762 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ec4acb46-c1b2-4356-b94a-11b128d10865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:29.768086 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:29.773280 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1409/2821] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:29.783993 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f731c-57da-4b4d-b9a3-e2a214f1c02e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:29.785871 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9f9f731c-57da-4b4d-b9a3-e2a214f1c02e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:29.844116 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f731c-57da-4b4d-b9a3-e2a214f1c02e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:29.847690 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e93259-097d-4d3a-912e-c2962db80cbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:29.848356 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e93259-097d-4d3a-912e-c2962db80cbf service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:29.848630 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e93259-097d-4d3a-912e-c2962db80cbf service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:29.848898 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e93259-097d-4d3a-912e-c2962db80cbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:29.857681 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e93259-097d-4d3a-912e-c2962db80cbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:29.859095 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e93259-097d-4d3a-912e-c2962db80cbf service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:29.868596 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1413/2822] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:48:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:48:29.908263 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4acb46-c1b2-4356-b94a-11b128d10865 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:29.909688 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ec4acb46-c1b2-4356-b94a-11b128d10865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:29.924882 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4acb46-c1b2-4356-b94a-11b128d10865 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:29.928632 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58998f-16df-4be2-8345-f351df20b2a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:29.929268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58998f-16df-4be2-8345-f351df20b2a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:29.929551 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58998f-16df-4be2-8345-f351df20b2a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:29.929828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58998f-16df-4be2-8345-f351df20b2a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:29.987799 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58998f-16df-4be2-8345-f351df20b2a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:29.989546 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58998f-16df-4be2-8345-f351df20b2a8 service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:29.998245 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1410/2823] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:48:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:48:32.839832 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e93259-097d-4d3a-912e-c2962db80cbf service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:32.901446 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e93259-097d-4d3a-912e-c2962db80cbf service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:32.905947 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58146a20-44e3-4b98-a6bf-21bd23c54494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:32.906797 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58146a20-44e3-4b98-a6bf-21bd23c54494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:32.907075 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58146a20-44e3-4b98-a6bf-21bd23c54494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:32.907345 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58146a20-44e3-4b98-a6bf-21bd23c54494 admin admin] PATH_INFO: `/v3/users/91d036196c1e4e07a4b13a334bbff646` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:32.908539 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58146a20-44e3-4b98-a6bf-21bd23c54494 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91d036196c1e4e07a4b13a334bbff646)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:32.910411 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58146a20-44e3-4b98-a6bf-21bd23c54494 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:32.912860 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:32.954597 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-58146a20-44e3-4b98-a6bf-21bd23c54494 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91d036196c1e4e07a4b13a334bbff646'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:32.961924 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-58146a20-44e3-4b98-a6bf-21bd23c54494 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91d036196c1e4e07a4b13a334bbff646'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:32.962596 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-58146a20-44e3-4b98-a6bf-21bd23c54494 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91d036196c1e4e07a4b13a334bbff646'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:32.963360 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-58146a20-44e3-4b98-a6bf-21bd23c54494 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91d036196c1e4e07a4b13a334bbff646'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:32.975368 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-58146a20-44e3-4b98-a6bf-21bd23c54494 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91d036196c1e4e07a4b13a334bbff646'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:32.987258 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1414/2824] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:48:32 2022] DELETE /identity/v3/users/91d036196c1e4e07a4b13a334bbff646 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:32.992011 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0fa0b-362a-44b1-a1eb-73b3f2e75e95 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:32.992549 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0fa0b-362a-44b1-a1eb-73b3f2e75e95 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:32.992549 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0fa0b-362a-44b1-a1eb-73b3f2e75e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:33.010846 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-6dd0fa0b-362a-44b1-a1eb-73b3f2e75e95 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:33.012113 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-6dd0fa0b-362a-44b1-a1eb-73b3f2e75e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:33.071045 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1411/2825] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:33.080828 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58146a20-44e3-4b98-a6bf-21bd23c54494 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:33.082233 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-58146a20-44e3-4b98-a6bf-21bd23c54494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:33.138611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58146a20-44e3-4b98-a6bf-21bd23c54494 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:33.142554 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44744c-7a02-456f-9b54-37435fda9015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:33.143187 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44744c-7a02-456f-9b54-37435fda9015 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:33.143474 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44744c-7a02-456f-9b54-37435fda9015 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:33.143751 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44744c-7a02-456f-9b54-37435fda9015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:33.152723 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44744c-7a02-456f-9b54-37435fda9015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:33.154237 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44744c-7a02-456f-9b54-37435fda9015 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:33.163884 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1415/2826] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:48:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:48:33.427899 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0fa0b-362a-44b1-a1eb-73b3f2e75e95 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:33.429343 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-6dd0fa0b-362a-44b1-a1eb-73b3f2e75e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:33.443439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0fa0b-362a-44b1-a1eb-73b3f2e75e95 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:33.448707 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77451d-319d-442f-afdd-e4d997e4df08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:33.449755 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77451d-319d-442f-afdd-e4d997e4df08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:33.450021 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77451d-319d-442f-afdd-e4d997e4df08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:33.450296 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77451d-319d-442f-afdd-e4d997e4df08 admin admin] PATH_INFO: `/v3/projects/d4c81fed3e1143edbb156e058f33b6ae` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:33.452004 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77451d-319d-442f-afdd-e4d997e4df08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4c81fed3e1143edbb156e058f33b6ae)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:33.453864 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77451d-319d-442f-afdd-e4d997e4df08 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:33.502136 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-ce77451d-319d-442f-afdd-e4d997e4df08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4c81fed3e1143edbb156e058f33b6ae', 'request_id': 'req-ce77451d-319d-442f-afdd-e4d997e4df08'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:33.508084 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-ce77451d-319d-442f-afdd-e4d997e4df08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4c81fed3e1143edbb156e058f33b6ae', 'request_id': 'req-ce77451d-319d-442f-afdd-e4d997e4df08'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:33.515725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-ce77451d-319d-442f-afdd-e4d997e4df08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4c81fed3e1143edbb156e058f33b6ae', 'request_id': 'req-ce77451d-319d-442f-afdd-e4d997e4df08'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:33.526707 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-ce77451d-319d-442f-afdd-e4d997e4df08 admin admin] The token cache is being invalidate because project d4c81fed3e1143edbb156e058f33b6ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:48:33.526992 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-ce77451d-319d-442f-afdd-e4d997e4df08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:33.527608 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-ce77451d-319d-442f-afdd-e4d997e4df08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:33.529157 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1412/2827] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:48:33 2022] DELETE /identity/v3/projects/d4c81fed3e1143edbb156e058f33b6ae => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:33.533101 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44744c-7a02-456f-9b54-37435fda9015 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:33.543018 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44744c-7a02-456f-9b54-37435fda9015 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:33.598470 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8254e3-e4a9-45b1-91f3-405949fe10a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:33.599405 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8254e3-e4a9-45b1-91f3-405949fe10a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:33.599685 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8254e3-e4a9-45b1-91f3-405949fe10a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:33.599982 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8254e3-e4a9-45b1-91f3-405949fe10a4 admin admin] PATH_INFO: `/v3/users/a31d151359d648899720bfc724c9695f` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:33.601163 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8254e3-e4a9-45b1-91f3-405949fe10a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a31d151359d648899720bfc724c9695f)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:33.603059 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8254e3-e4a9-45b1-91f3-405949fe10a4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:33.630187 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:33.649178 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8e8254e3-e4a9-45b1-91f3-405949fe10a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a31d151359d648899720bfc724c9695f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:33.655872 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8e8254e3-e4a9-45b1-91f3-405949fe10a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a31d151359d648899720bfc724c9695f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:33.656535 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8e8254e3-e4a9-45b1-91f3-405949fe10a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a31d151359d648899720bfc724c9695f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:33.657132 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8e8254e3-e4a9-45b1-91f3-405949fe10a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a31d151359d648899720bfc724c9695f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:33.682182 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8e8254e3-e4a9-45b1-91f3-405949fe10a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a31d151359d648899720bfc724c9695f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:33.696708 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1416/2828] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:48:33 2022] DELETE /identity/v3/users/a31d151359d648899720bfc724c9695f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:33.978611 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77451d-319d-442f-afdd-e4d997e4df08 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:33.980071 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ce77451d-319d-442f-afdd-e4d997e4df08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:33.993726 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77451d-319d-442f-afdd-e4d997e4df08 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:34.017244 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:34.059141 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10b459-5945-4e51-b698-8651bf182e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:34.059957 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10b459-5945-4e51-b698-8651bf182e86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:34.060245 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10b459-5945-4e51-b698-8651bf182e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:34.060525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10b459-5945-4e51-b698-8651bf182e86 admin admin] PATH_INFO: `/v3/projects/1f4801c2c9104201919853c36aab938e` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:34.061702 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b10b459-5945-4e51-b698-8651bf182e86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f4801c2c9104201919853c36aab938e)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:34.063605 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b10b459-5945-4e51-b698-8651bf182e86 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:34.110484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3b10b459-5945-4e51-b698-8651bf182e86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f4801c2c9104201919853c36aab938e', 'request_id': 'req-3b10b459-5945-4e51-b698-8651bf182e86'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:34.116144 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3b10b459-5945-4e51-b698-8651bf182e86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f4801c2c9104201919853c36aab938e', 'request_id': 'req-3b10b459-5945-4e51-b698-8651bf182e86'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:34.124102 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3b10b459-5945-4e51-b698-8651bf182e86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f4801c2c9104201919853c36aab938e', 'request_id': 'req-3b10b459-5945-4e51-b698-8651bf182e86'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:34.136039 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3b10b459-5945-4e51-b698-8651bf182e86 admin admin] The token cache is being invalidate because project 1f4801c2c9104201919853c36aab938e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:48:34.136358 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3b10b459-5945-4e51-b698-8651bf182e86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:34.136991 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3b10b459-5945-4e51-b698-8651bf182e86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:34.138542 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1413/2829] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:48:33 2022] DELETE /identity/v3/projects/1f4801c2c9104201919853c36aab938e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:34.142751 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8254e3-e4a9-45b1-91f3-405949fe10a4 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:34.144195 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-8e8254e3-e4a9-45b1-91f3-405949fe10a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:34.157623 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8254e3-e4a9-45b1-91f3-405949fe10a4 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:34.213008 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d5547-c10e-4666-b96e-bc8791b030ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:34.213875 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d5547-c10e-4666-b96e-bc8791b030ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:34.214161 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d5547-c10e-4666-b96e-bc8791b030ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:34.214442 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d5547-c10e-4666-b96e-bc8791b030ed admin admin] PATH_INFO: `/v3/users/87243418818342cc84a4967b6cfcf3d9` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:34.215624 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d5547-c10e-4666-b96e-bc8791b030ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=87243418818342cc84a4967b6cfcf3d9)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:34.217569 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d5547-c10e-4666-b96e-bc8791b030ed admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:34.261996 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a58d5547-c10e-4666-b96e-bc8791b030ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87243418818342cc84a4967b6cfcf3d9'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:34.268995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a58d5547-c10e-4666-b96e-bc8791b030ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87243418818342cc84a4967b6cfcf3d9'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:34.270414 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a58d5547-c10e-4666-b96e-bc8791b030ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87243418818342cc84a4967b6cfcf3d9'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:34.271128 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a58d5547-c10e-4666-b96e-bc8791b030ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87243418818342cc84a4967b6cfcf3d9'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:34.283120 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a58d5547-c10e-4666-b96e-bc8791b030ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87243418818342cc84a4967b6cfcf3d9'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:34.295851 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1417/2830] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:48:34 2022] DELETE /identity/v3/users/87243418818342cc84a4967b6cfcf3d9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:34.578864 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10b459-5945-4e51-b698-8651bf182e86 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:34.580445 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-3b10b459-5945-4e51-b698-8651bf182e86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:34.596351 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10b459-5945-4e51-b698-8651bf182e86 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:34.655116 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22d05f-f874-4dca-a7e0-e3e85e3f3bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:34.656005 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22d05f-f874-4dca-a7e0-e3e85e3f3bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:34.656291 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22d05f-f874-4dca-a7e0-e3e85e3f3bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:34.656566 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22d05f-f874-4dca-a7e0-e3e85e3f3bac admin admin] PATH_INFO: `/v3/projects/6bf903d8a2694c3d892491002b1d7df6` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:34.657746 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22d05f-f874-4dca-a7e0-e3e85e3f3bac admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bf903d8a2694c3d892491002b1d7df6)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:34.659637 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22d05f-f874-4dca-a7e0-e3e85e3f3bac admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:34.705319 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7e22d05f-f874-4dca-a7e0-e3e85e3f3bac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bf903d8a2694c3d892491002b1d7df6', 'request_id': 'req-7e22d05f-f874-4dca-a7e0-e3e85e3f3bac'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:34.709290 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:34.711514 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7e22d05f-f874-4dca-a7e0-e3e85e3f3bac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bf903d8a2694c3d892491002b1d7df6', 'request_id': 'req-7e22d05f-f874-4dca-a7e0-e3e85e3f3bac'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:34.720111 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7e22d05f-f874-4dca-a7e0-e3e85e3f3bac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bf903d8a2694c3d892491002b1d7df6', 'request_id': 'req-7e22d05f-f874-4dca-a7e0-e3e85e3f3bac'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:34.731445 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7e22d05f-f874-4dca-a7e0-e3e85e3f3bac admin admin] The token cache is being invalidate because project 6bf903d8a2694c3d892491002b1d7df6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:48:34.731734 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7e22d05f-f874-4dca-a7e0-e3e85e3f3bac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:34.732393 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-7e22d05f-f874-4dca-a7e0-e3e85e3f3bac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:34.733921 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1414/2831] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:48:34 2022] DELETE /identity/v3/projects/6bf903d8a2694c3d892491002b1d7df6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:34.740835 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90ed73-faf5-408e-911e-3b0e9f0feba8 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:34.741129 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90ed73-faf5-408e-911e-3b0e9f0feba8 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:34.741409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90ed73-faf5-408e-911e-3b0e9f0feba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:34.759342 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-fa90ed73-faf5-408e-911e-3b0e9f0feba8 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:34.760565 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fa90ed73-faf5-408e-911e-3b0e9f0feba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:34.777487 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:34.823644 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1418/2832] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:34.831935 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a8e1a-45f1-4dbf-a465-e101d83ebcd7 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:34.832280 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a8e1a-45f1-4dbf-a465-e101d83ebcd7 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:34.832552 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a8e1a-45f1-4dbf-a465-e101d83ebcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:34.850290 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-920a8e1a-45f1-4dbf-a465-e101d83ebcd7 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:34.851477 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-920a8e1a-45f1-4dbf-a465-e101d83ebcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:34.859229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1415/2833] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:34.863830 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90ed73-faf5-408e-911e-3b0e9f0feba8 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:34.873622 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90ed73-faf5-408e-911e-3b0e9f0feba8 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:34.878505 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fa43a-dc9a-4907-9c33-59a0aec8827e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:34.879918 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fa43a-dc9a-4907-9c33-59a0aec8827e admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:34.880229 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fa43a-dc9a-4907-9c33-59a0aec8827e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:34.880497 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fa43a-dc9a-4907-9c33-59a0aec8827e admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:34.881214 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fa43a-dc9a-4907-9c33-59a0aec8827e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:48:34.881518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fa43a-dc9a-4907-9c33-59a0aec8827e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:34.883100 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fa43a-dc9a-4907-9c33-59a0aec8827e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:34.895284 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1419/2834] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:48:34.899589 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a8e1a-45f1-4dbf-a465-e101d83ebcd7 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:34.909006 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a8e1a-45f1-4dbf-a465-e101d83ebcd7 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:34.914550 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecff72-f4b8-4f26-a486-33863a861661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:34.915207 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecff72-f4b8-4f26-a486-33863a861661 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:34.915500 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecff72-f4b8-4f26-a486-33863a861661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:34.915777 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecff72-f4b8-4f26-a486-33863a861661 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:34.916509 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ecff72-f4b8-4f26-a486-33863a861661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:34.918398 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ecff72-f4b8-4f26-a486-33863a861661 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:34.945867 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1416/2835] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:48:34 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:48:34.949498 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fa43a-dc9a-4907-9c33-59a0aec8827e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:34.960247 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fa43a-dc9a-4907-9c33-59a0aec8827e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:35.016125 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67547283-e62f-45ed-853f-b3ffa4947330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:35.016792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67547283-e62f-45ed-853f-b3ffa4947330 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:35.017072 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67547283-e62f-45ed-853f-b3ffa4947330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:35.017343 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67547283-e62f-45ed-853f-b3ffa4947330 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:35.018032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67547283-e62f-45ed-853f-b3ffa4947330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:35.020024 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67547283-e62f-45ed-853f-b3ffa4947330 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:35.039965 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1420/2836] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:48:34 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:48:35.044247 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecff72-f4b8-4f26-a486-33863a861661 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:35.055353 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecff72-f4b8-4f26-a486-33863a861661 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:35.059981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb575e73-5f0c-4f52-93a5-20a7a3368d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:35.060634 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb575e73-5f0c-4f52-93a5-20a7a3368d14 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:35.060922 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb575e73-5f0c-4f52-93a5-20a7a3368d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:35.061201 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb575e73-5f0c-4f52-93a5-20a7a3368d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:35.061953 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb575e73-5f0c-4f52-93a5-20a7a3368d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:35.063540 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb575e73-5f0c-4f52-93a5-20a7a3368d14 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:35.074102 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1417/2837] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:48:35.077942 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67547283-e62f-45ed-853f-b3ffa4947330 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:35.087998 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67547283-e62f-45ed-853f-b3ffa4947330 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:35.092623 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610166f6-b4b1-4bd8-8e39-617df1c59973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:35.093275 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610166f6-b4b1-4bd8-8e39-617df1c59973 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:35.093558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610166f6-b4b1-4bd8-8e39-617df1c59973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:35.093837 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610166f6-b4b1-4bd8-8e39-617df1c59973 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:35.094551 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610166f6-b4b1-4bd8-8e39-617df1c59973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:35.096192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610166f6-b4b1-4bd8-8e39-617df1c59973 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:35.106475 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1421/2838] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:48:35.110644 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb575e73-5f0c-4f52-93a5-20a7a3368d14 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:35.121726 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb575e73-5f0c-4f52-93a5-20a7a3368d14 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:35.126309 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506876c-489d-4108-95fa-15d47466245d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:35.127176 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506876c-489d-4108-95fa-15d47466245d admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:35.127458 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506876c-489d-4108-95fa-15d47466245d admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:35.127733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506876c-489d-4108-95fa-15d47466245d admin admin] PATH_INFO: `/v3/projects/9f055f4c69ee4ac281ccc939509faa6e/users/a63e1d72ff9348d7999c18c03712f65c/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:35.143284 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b506876c-489d-4108-95fa-15d47466245d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f055f4c69ee4ac281ccc939509faa6e, user_id=a63e1d72ff9348d7999c18c03712f65c, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:35.145971 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b506876c-489d-4108-95fa-15d47466245d admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:35.159820 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1418/2839] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:48:35 2022] PUT /identity/v3/projects/9f055f4c69ee4ac281ccc939509faa6e/users/a63e1d72ff9348d7999c18c03712f65c/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:35.171642 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b733f-810f-4501-a203-ae8bbca77292 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:35.172168 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b733f-810f-4501-a203-ae8bbca77292 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:35.172276 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b733f-810f-4501-a203-ae8bbca77292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:35.198518 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ef9b733f-810f-4501-a203-ae8bbca77292 None None] MFA Rules not processed for user `a63e1d72ff9348d7999c18c03712f65c`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:35.199719 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ef9b733f-810f-4501-a203-ae8bbca77292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:35.263065 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1422/2840] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:35.275353 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506876c-489d-4108-95fa-15d47466245d admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:35.277039 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-b506876c-489d-4108-95fa-15d47466245d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:35.335766 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506876c-489d-4108-95fa-15d47466245d admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:35.339382 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5710ad0-59e1-4361-93fa-928a8a8b807b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:35.340053 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5710ad0-59e1-4361-93fa-928a8a8b807b service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:35.340340 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5710ad0-59e1-4361-93fa-928a8a8b807b service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:35.340617 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5710ad0-59e1-4361-93fa-928a8a8b807b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:35.349576 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5710ad0-59e1-4361-93fa-928a8a8b807b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:35.350993 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5710ad0-59e1-4361-93fa-928a8a8b807b service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:35.359875 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1419/2841] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:48:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:48:37.900918 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b733f-810f-4501-a203-ae8bbca77292 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:37.920359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:37.966141 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b733f-810f-4501-a203-ae8bbca77292 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:37.970622 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f7edc-2d0a-471f-bc54-3a4bf9233bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:37.971504 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f7edc-2d0a-471f-bc54-3a4bf9233bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:37.971774 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f7edc-2d0a-471f-bc54-3a4bf9233bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:37.972123 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f7edc-2d0a-471f-bc54-3a4bf9233bde admin admin] PATH_INFO: `/v3/users/a63e1d72ff9348d7999c18c03712f65c` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:37.974015 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f7edc-2d0a-471f-bc54-3a4bf9233bde admin admin] RBAC: Authorizing `identity:delete_user(user_id=a63e1d72ff9348d7999c18c03712f65c)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:37.975974 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f7edc-2d0a-471f-bc54-3a4bf9233bde admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:38.022745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-077f7edc-2d0a-471f-bc54-3a4bf9233bde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a63e1d72ff9348d7999c18c03712f65c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:38.029430 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-077f7edc-2d0a-471f-bc54-3a4bf9233bde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a63e1d72ff9348d7999c18c03712f65c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:38.030083 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-077f7edc-2d0a-471f-bc54-3a4bf9233bde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a63e1d72ff9348d7999c18c03712f65c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:38.030712 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-077f7edc-2d0a-471f-bc54-3a4bf9233bde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a63e1d72ff9348d7999c18c03712f65c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:38.042770 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-077f7edc-2d0a-471f-bc54-3a4bf9233bde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a63e1d72ff9348d7999c18c03712f65c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:38.056091 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1423/2842] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:48:37 2022] DELETE /identity/v3/users/a63e1d72ff9348d7999c18c03712f65c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:38.060725 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb3efc-2bd5-4fb4-81ef-b7fa9486f01f None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:38.061223 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb3efc-2bd5-4fb4-81ef-b7fa9486f01f None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:38.061345 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb3efc-2bd5-4fb4-81ef-b7fa9486f01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:38.079354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-5fcb3efc-2bd5-4fb4-81ef-b7fa9486f01f None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:38.080595 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-5fcb3efc-2bd5-4fb4-81ef-b7fa9486f01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:38.138036 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1420/2843] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:38.148096 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f7edc-2d0a-471f-bc54-3a4bf9233bde admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:38.149525 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-077f7edc-2d0a-471f-bc54-3a4bf9233bde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:38.206127 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f7edc-2d0a-471f-bc54-3a4bf9233bde admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:38.209866 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464aa6c-e511-4ed4-995a-9e8156f4bb7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:38.210502 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464aa6c-e511-4ed4-995a-9e8156f4bb7e service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:38.210786 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464aa6c-e511-4ed4-995a-9e8156f4bb7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:38.211057 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464aa6c-e511-4ed4-995a-9e8156f4bb7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:38.219959 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d464aa6c-e511-4ed4-995a-9e8156f4bb7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:38.221384 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d464aa6c-e511-4ed4-995a-9e8156f4bb7e service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:38.231390 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1424/2844] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:48:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:48:38.499505 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb3efc-2bd5-4fb4-81ef-b7fa9486f01f None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:38.501003 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-5fcb3efc-2bd5-4fb4-81ef-b7fa9486f01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:38.516556 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb3efc-2bd5-4fb4-81ef-b7fa9486f01f None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:38.522837 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d79e7b-af7d-4cc1-9171-9721f4dc9ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:38.523743 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d79e7b-af7d-4cc1-9171-9721f4dc9ccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:38.524011 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d79e7b-af7d-4cc1-9171-9721f4dc9ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:38.524289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d79e7b-af7d-4cc1-9171-9721f4dc9ccb admin admin] PATH_INFO: `/v3/projects/9f055f4c69ee4ac281ccc939509faa6e` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:38.525592 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d79e7b-af7d-4cc1-9171-9721f4dc9ccb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f055f4c69ee4ac281ccc939509faa6e)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:38.527535 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d79e7b-af7d-4cc1-9171-9721f4dc9ccb admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:38.545268 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:38.576516 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-f2d79e7b-af7d-4cc1-9171-9721f4dc9ccb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f055f4c69ee4ac281ccc939509faa6e', 'request_id': 'req-f2d79e7b-af7d-4cc1-9171-9721f4dc9ccb'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:38.582422 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-f2d79e7b-af7d-4cc1-9171-9721f4dc9ccb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f055f4c69ee4ac281ccc939509faa6e', 'request_id': 'req-f2d79e7b-af7d-4cc1-9171-9721f4dc9ccb'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:38.589440 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-f2d79e7b-af7d-4cc1-9171-9721f4dc9ccb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f055f4c69ee4ac281ccc939509faa6e', 'request_id': 'req-f2d79e7b-af7d-4cc1-9171-9721f4dc9ccb'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:38.600327 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-f2d79e7b-af7d-4cc1-9171-9721f4dc9ccb admin admin] The token cache is being invalidate because project 9f055f4c69ee4ac281ccc939509faa6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:48:38.600613 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-f2d79e7b-af7d-4cc1-9171-9721f4dc9ccb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:38.601229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-f2d79e7b-af7d-4cc1-9171-9721f4dc9ccb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:48:38.602779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1421/2845] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:48:38 2022] DELETE /identity/v3/projects/9f055f4c69ee4ac281ccc939509faa6e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:38.607755 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700cc14-3788-46a0-8261-ef654c04129d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:38.608075 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700cc14-3788-46a0-8261-ef654c04129d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:38.608348 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700cc14-3788-46a0-8261-ef654c04129d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:38.626036 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-e700cc14-3788-46a0-8261-ef654c04129d None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:38.627307 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e700cc14-3788-46a0-8261-ef654c04129d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:38.664239 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:38.691119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1425/2846] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:38.699435 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941b642-a6f1-49e4-87b0-702b316b59f3 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:38.699435 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941b642-a6f1-49e4-87b0-702b316b59f3 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:38.699435 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941b642-a6f1-49e4-87b0-702b316b59f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:38.719031 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-3941b642-a6f1-49e4-87b0-702b316b59f3 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:38.721118 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-3941b642-a6f1-49e4-87b0-702b316b59f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:38.731260 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1422/2847] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:38.735465 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e700cc14-3788-46a0-8261-ef654c04129d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:38.746649 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e700cc14-3788-46a0-8261-ef654c04129d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:38.751203 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515b962-91ba-485f-8d37-cb6e8cfab42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:38.752243 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515b962-91ba-485f-8d37-cb6e8cfab42a admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:38.752528 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515b962-91ba-485f-8d37-cb6e8cfab42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:38.752800 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515b962-91ba-485f-8d37-cb6e8cfab42a admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:38.753517 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515b962-91ba-485f-8d37-cb6e8cfab42a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:48:38.753898 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515b962-91ba-485f-8d37-cb6e8cfab42a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:38.756031 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515b962-91ba-485f-8d37-cb6e8cfab42a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:38.767566 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1426/2848] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 10:48:38.772047 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941b642-a6f1-49e4-87b0-702b316b59f3 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:38.781591 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941b642-a6f1-49e4-87b0-702b316b59f3 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:38.786929 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06ed17-3e2e-432e-a420-a84fdc7585f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:38.787686 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06ed17-3e2e-432e-a420-a84fdc7585f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:38.788009 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06ed17-3e2e-432e-a420-a84fdc7585f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:38.788309 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06ed17-3e2e-432e-a420-a84fdc7585f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:38.788999 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06ed17-3e2e-432e-a420-a84fdc7585f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:38.790920 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06ed17-3e2e-432e-a420-a84fdc7585f7 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:38.816938 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1423/2849] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:48:38 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:48:38.820943 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515b962-91ba-485f-8d37-cb6e8cfab42a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:38.830787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515b962-91ba-485f-8d37-cb6e8cfab42a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:38.886198 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12cb54-8afa-4e4f-bf56-54408f560203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:38.886855 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12cb54-8afa-4e4f-bf56-54408f560203 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:38.887136 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12cb54-8afa-4e4f-bf56-54408f560203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:38.887413 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12cb54-8afa-4e4f-bf56-54408f560203 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:38.888138 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b12cb54-8afa-4e4f-bf56-54408f560203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:38.890071 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b12cb54-8afa-4e4f-bf56-54408f560203 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:38.909964 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1427/2850] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:48:38 2022] POST /identity/v3/users => generated 285 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:48:38.914893 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06ed17-3e2e-432e-a420-a84fdc7585f7 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:38.925061 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06ed17-3e2e-432e-a420-a84fdc7585f7 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:38.930132 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36672a-161b-4b71-996e-a77f865b9a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:38.930779 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36672a-161b-4b71-996e-a77f865b9a66 admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:38.931060 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36672a-161b-4b71-996e-a77f865b9a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:38.931332 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36672a-161b-4b71-996e-a77f865b9a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:38.932079 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36672a-161b-4b71-996e-a77f865b9a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:38.933649 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36672a-161b-4b71-996e-a77f865b9a66 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:38.943973 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1424/2851] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:48:38.947511 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12cb54-8afa-4e4f-bf56-54408f560203 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:38.957836 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12cb54-8afa-4e4f-bf56-54408f560203 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:38.962394 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516299a4-4e4c-4383-bda5-034d773036b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:38.963028 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516299a4-4e4c-4383-bda5-034d773036b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:38.963308 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516299a4-4e4c-4383-bda5-034d773036b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:38.963577 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516299a4-4e4c-4383-bda5-034d773036b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:38.964408 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516299a4-4e4c-4383-bda5-034d773036b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:38.966319 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516299a4-4e4c-4383-bda5-034d773036b1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:38.976912 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1428/2852] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:48:38.981023 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36672a-161b-4b71-996e-a77f865b9a66 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:38.991852 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36672a-161b-4b71-996e-a77f865b9a66 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:38.996496 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e784d4-75f5-4207-b4ea-31196b6ed7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:38.997370 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e784d4-75f5-4207-b4ea-31196b6ed7f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:38.997648 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e784d4-75f5-4207-b4ea-31196b6ed7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:38.997919 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e784d4-75f5-4207-b4ea-31196b6ed7f3 admin admin] PATH_INFO: `/v3/projects/f4a14d71334b4102a5c8255a38c18189/users/8f48f277b7444cddb0677cde50893a0a/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:39.013402 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e784d4-75f5-4207-b4ea-31196b6ed7f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4a14d71334b4102a5c8255a38c18189, user_id=8f48f277b7444cddb0677cde50893a0a, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:39.016338 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e784d4-75f5-4207-b4ea-31196b6ed7f3 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:39.029850 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1425/2853] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:48:38 2022] PUT /identity/v3/projects/f4a14d71334b4102a5c8255a38c18189/users/8f48f277b7444cddb0677cde50893a0a/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:48:39.041075 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29370e-2dec-46f9-9838-d0c10ad3cf98 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:39.041392 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29370e-2dec-46f9-9838-d0c10ad3cf98 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:39.041676 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29370e-2dec-46f9-9838-d0c10ad3cf98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:39.067860 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9c29370e-2dec-46f9-9838-d0c10ad3cf98 None None] MFA Rules not processed for user `8f48f277b7444cddb0677cde50893a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:48:39.069066 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9c29370e-2dec-46f9-9838-d0c10ad3cf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:39.133021 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1429/2854] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:48:39 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:48:39.145501 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e784d4-75f5-4207-b4ea-31196b6ed7f3 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:48:39.147167 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-81e784d4-75f5-4207-b4ea-31196b6ed7f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:48:39.202323 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e784d4-75f5-4207-b4ea-31196b6ed7f3 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:48:39.205849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c17689-b4c5-4801-973b-15fcb9bee91a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a67e007fc314b34aac2471a7c20f240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:48:39.206480 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c17689-b4c5-4801-973b-15fcb9bee91a service glance] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:48:39.206760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c17689-b4c5-4801-973b-15fcb9bee91a service glance] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:48:39.207535 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c17689-b4c5-4801-973b-15fcb9bee91a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:48:39.216267 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c17689-b4c5-4801-973b-15fcb9bee91a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:48:39.217709 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c17689-b4c5-4801-973b-15fcb9bee91a service glance] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:48:39.227289 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1426/2855] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:48:39 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:48:49.130060 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:48:49.222502 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:49:03.770932 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29370e-2dec-46f9-9838-d0c10ad3cf98 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:49:03.837084 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29370e-2dec-46f9-9838-d0c10ad3cf98 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:49:03.841567 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b16676-49e6-4a00-9d88-d4bf80ee2659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:49:03.842432 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b16676-49e6-4a00-9d88-d4bf80ee2659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:03.842711 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b16676-49e6-4a00-9d88-d4bf80ee2659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:03.842980 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b16676-49e6-4a00-9d88-d4bf80ee2659 admin admin] PATH_INFO: `/v3/users/8f48f277b7444cddb0677cde50893a0a` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:03.844164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b16676-49e6-4a00-9d88-d4bf80ee2659 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f48f277b7444cddb0677cde50893a0a)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:49:03.846210 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b16676-49e6-4a00-9d88-d4bf80ee2659 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:49:03.890852 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-68b16676-49e6-4a00-9d88-d4bf80ee2659 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f48f277b7444cddb0677cde50893a0a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:49:03.899619 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-68b16676-49e6-4a00-9d88-d4bf80ee2659 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f48f277b7444cddb0677cde50893a0a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:49:03.900301 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-68b16676-49e6-4a00-9d88-d4bf80ee2659 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f48f277b7444cddb0677cde50893a0a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:49:03.900903 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-68b16676-49e6-4a00-9d88-d4bf80ee2659 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f48f277b7444cddb0677cde50893a0a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:49:03.913698 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-68b16676-49e6-4a00-9d88-d4bf80ee2659 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f48f277b7444cddb0677cde50893a0a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:49:03.926355 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1430/2856] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:49:03 2022] DELETE /identity/v3/users/8f48f277b7444cddb0677cde50893a0a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:49:03.930996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f1d29-7c49-4545-a9ea-66d96a910b69 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:03.931515 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f1d29-7c49-4545-a9ea-66d96a910b69 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:03.931625 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f1d29-7c49-4545-a9ea-66d96a910b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:03.948215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-b75f1d29-7c49-4545-a9ea-66d96a910b69 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:49:03.949405 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-b75f1d29-7c49-4545-a9ea-66d96a910b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:49:04.007979 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1427/2857] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:49:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:49:04.017886 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b16676-49e6-4a00-9d88-d4bf80ee2659 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:49:04.019310 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-68b16676-49e6-4a00-9d88-d4bf80ee2659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:49:04.075916 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b16676-49e6-4a00-9d88-d4bf80ee2659 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:49:04.079552 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb633654-8a69-4197-bd2d-6d259677d1d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:49:04.080223 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb633654-8a69-4197-bd2d-6d259677d1d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:04.080511 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb633654-8a69-4197-bd2d-6d259677d1d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:04.080792 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb633654-8a69-4197-bd2d-6d259677d1d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:04.090653 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb633654-8a69-4197-bd2d-6d259677d1d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:49:04.092152 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb633654-8a69-4197-bd2d-6d259677d1d2 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:49:04.103032 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1431/2858] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:49:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:49:04.377072 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f1d29-7c49-4545-a9ea-66d96a910b69 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:49:04.378572 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-b75f1d29-7c49-4545-a9ea-66d96a910b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:49:04.392771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f1d29-7c49-4545-a9ea-66d96a910b69 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:49:04.398365 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8887abd-79a3-4510-9014-8938b95c8c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:49:04.399249 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8887abd-79a3-4510-9014-8938b95c8c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:04.399534 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8887abd-79a3-4510-9014-8938b95c8c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:04.399872 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8887abd-79a3-4510-9014-8938b95c8c2d admin admin] PATH_INFO: `/v3/projects/f4a14d71334b4102a5c8255a38c18189` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:04.401073 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8887abd-79a3-4510-9014-8938b95c8c2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4a14d71334b4102a5c8255a38c18189)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:49:04.402986 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8887abd-79a3-4510-9014-8938b95c8c2d admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:49:04.450940 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b8887abd-79a3-4510-9014-8938b95c8c2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4a14d71334b4102a5c8255a38c18189', 'request_id': 'req-b8887abd-79a3-4510-9014-8938b95c8c2d'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:49:04.456423 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b8887abd-79a3-4510-9014-8938b95c8c2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4a14d71334b4102a5c8255a38c18189', 'request_id': 'req-b8887abd-79a3-4510-9014-8938b95c8c2d'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:49:04.466427 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b8887abd-79a3-4510-9014-8938b95c8c2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4a14d71334b4102a5c8255a38c18189', 'request_id': 'req-b8887abd-79a3-4510-9014-8938b95c8c2d'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:49:04.479528 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b8887abd-79a3-4510-9014-8938b95c8c2d admin admin] The token cache is being invalidate because project f4a14d71334b4102a5c8255a38c18189 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:49:04.479849 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b8887abd-79a3-4510-9014-8938b95c8c2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:49:04.480503 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-b8887abd-79a3-4510-9014-8938b95c8c2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:49:04.482053 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1428/2859] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:49:04 2022] DELETE /identity/v3/projects/f4a14d71334b4102a5c8255a38c18189 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:49:04.487152 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae7841-ee84-46a3-a8d2-dd313c30305e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:04.487447 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae7841-ee84-46a3-a8d2-dd313c30305e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:04.487725 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae7841-ee84-46a3-a8d2-dd313c30305e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:04.505669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3dae7841-ee84-46a3-a8d2-dd313c30305e None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:49:04.506869 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3dae7841-ee84-46a3-a8d2-dd313c30305e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:49:04.524990 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:49:04.569427 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1432/2860] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:49:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:49:04.578806 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f225f0-7bd4-4f27-bee8-b5bc61ef22e2 None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:04.579102 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f225f0-7bd4-4f27-bee8-b5bc61ef22e2 None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:04.579383 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f225f0-7bd4-4f27-bee8-b5bc61ef22e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:04.597749 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-50f225f0-7bd4-4f27-bee8-b5bc61ef22e2 None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:49:04.598962 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-50f225f0-7bd4-4f27-bee8-b5bc61ef22e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:49:04.606853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1429/2861] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:49:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:49:04.611099 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae7841-ee84-46a3-a8d2-dd313c30305e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:49:04.621255 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae7841-ee84-46a3-a8d2-dd313c30305e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:49:04.625839 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57d400-aabd-4744-8483-c4bf366dcf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:49:04.626488 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57d400-aabd-4744-8483-c4bf366dcf16 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:04.626774 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57d400-aabd-4744-8483-c4bf366dcf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:04.627051 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57d400-aabd-4744-8483-c4bf366dcf16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:04.627813 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57d400-aabd-4744-8483-c4bf366dcf16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 10:49:04.628117 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57d400-aabd-4744-8483-c4bf366dcf16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:49:04.630303 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57d400-aabd-4744-8483-c4bf366dcf16 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:49:04.642017 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1433/2862] 10.222.0.42 () {58 vars in 1152 bytes} [Mon Jun 6 10:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 10:49:04.646137 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f225f0-7bd4-4f27-bee8-b5bc61ef22e2 None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:49:04.655215 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f225f0-7bd4-4f27-bee8-b5bc61ef22e2 None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:49:04.659798 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7983848-3109-447a-8caa-e39b604dcdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:49:04.660720 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7983848-3109-447a-8caa-e39b604dcdf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:04.660994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7983848-3109-447a-8caa-e39b604dcdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:04.661276 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7983848-3109-447a-8caa-e39b604dcdf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:04.661967 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7983848-3109-447a-8caa-e39b604dcdf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:49:04.663860 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7983848-3109-447a-8caa-e39b604dcdf8 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:49:04.688525 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1430/2863] 10.222.0.42 () {60 vars in 1152 bytes} [Mon Jun 6 10:49:04 2022] POST /identity/v3/projects => generated 399 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:49:04.693100 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57d400-aabd-4744-8483-c4bf366dcf16 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:49:04.703205 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57d400-aabd-4744-8483-c4bf366dcf16 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:49:04.760294 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cb63a-45f0-4c08-b069-69c642ed1d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:49:04.760966 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cb63a-45f0-4c08-b069-69c642ed1d66 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:04.761251 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cb63a-45f0-4c08-b069-69c642ed1d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:04.761528 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cb63a-45f0-4c08-b069-69c642ed1d66 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:04.762409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97cb63a-45f0-4c08-b069-69c642ed1d66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:49:04.764381 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97cb63a-45f0-4c08-b069-69c642ed1d66 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:49:04.786950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1434/2864] 10.222.0.42 () {60 vars in 1143 bytes} [Mon Jun 6 10:49:04 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 10:49:04.790972 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7983848-3109-447a-8caa-e39b604dcdf8 admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:49:04.802135 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7983848-3109-447a-8caa-e39b604dcdf8 admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:49:04.806716 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013319ee-e87d-40bf-94c2-575e169534be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:49:04.807399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013319ee-e87d-40bf-94c2-575e169534be admin admin] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:04.807759 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013319ee-e87d-40bf-94c2-575e169534be admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:04.808066 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013319ee-e87d-40bf-94c2-575e169534be admin admin] PATH_INFO: `/v3/roles` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:04.808787 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013319ee-e87d-40bf-94c2-575e169534be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:49:04.810399 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013319ee-e87d-40bf-94c2-575e169534be admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:49:04.823094 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1431/2865] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 10:49:04.826765 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cb63a-45f0-4c08-b069-69c642ed1d66 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:49:04.837223 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cb63a-45f0-4c08-b069-69c642ed1d66 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:49:04.841941 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6552d2-1dee-40c9-9698-ae78ecb64922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:49:04.842594 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6552d2-1dee-40c9-9698-ae78ecb64922 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:04.842883 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6552d2-1dee-40c9-9698-ae78ecb64922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:04.843164 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6552d2-1dee-40c9-9698-ae78ecb64922 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:04.843922 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6552d2-1dee-40c9-9698-ae78ecb64922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:49:04.845520 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6552d2-1dee-40c9-9698-ae78ecb64922 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:49:04.855432 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1435/2866] 10.222.0.42 () {58 vars in 1121 bytes} [Mon Jun 6 10:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 10:49:04.859361 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013319ee-e87d-40bf-94c2-575e169534be admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:49:04.869747 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013319ee-e87d-40bf-94c2-575e169534be admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:49:04.874346 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa1488-66a0-4b7b-b3f9-cace1c2fb43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:49:04.875213 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa1488-66a0-4b7b-b3f9-cace1c2fb43e admin admin] REQUEST_METHOD: `PUT` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:04.875495 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa1488-66a0-4b7b-b3f9-cace1c2fb43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:04.875771 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa1488-66a0-4b7b-b3f9-cace1c2fb43e admin admin] PATH_INFO: `/v3/projects/bb6fbb13fad9446196f3ea81383f160c/users/27715ccd9ddd42ffbd8bb4b306ef4a0f/roles/6bcb06bf4aa4442ebcfebeadf1af41de` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:04.892229 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa1488-66a0-4b7b-b3f9-cace1c2fb43e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb6fbb13fad9446196f3ea81383f160c, user_id=27715ccd9ddd42ffbd8bb4b306ef4a0f, role_id=6bcb06bf4aa4442ebcfebeadf1af41de)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:49:04.895313 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa1488-66a0-4b7b-b3f9-cace1c2fb43e admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:49:04.908042 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1432/2867] 10.222.0.42 () {60 vars in 1482 bytes} [Mon Jun 6 10:49:04 2022] PUT /identity/v3/projects/bb6fbb13fad9446196f3ea81383f160c/users/27715ccd9ddd42ffbd8bb4b306ef4a0f/roles/6bcb06bf4aa4442ebcfebeadf1af41de => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:49:04.919531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19442a-720f-4def-90bd-c4b3b560157b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:04.919847 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19442a-720f-4def-90bd-c4b3b560157b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:04.920128 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19442a-720f-4def-90bd-c4b3b560157b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:04.948688 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3d19442a-720f-4def-90bd-c4b3b560157b None None] MFA Rules not processed for user `27715ccd9ddd42ffbd8bb4b306ef4a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:49:04.949922 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3d19442a-720f-4def-90bd-c4b3b560157b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:49:05.012791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1436/2868] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:49:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:49:05.024623 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa1488-66a0-4b7b-b3f9-cace1c2fb43e admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:49:05.026344 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a9aa1488-66a0-4b7b-b3f9-cace1c2fb43e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:49:05.048462 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:49:05.096458 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa1488-66a0-4b7b-b3f9-cace1c2fb43e admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:49:05.102490 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2b764-a285-4121-ba94-e019bf0597b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8836193f1a47e1a62ea5167e01b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:49:05.103159 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2b764-a285-4121-ba94-e019bf0597b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:05.103429 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2b764-a285-4121-ba94-e019bf0597b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:05.103700 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2b764-a285-4121-ba94-e019bf0597b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:05.112716 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2b764-a285-4121-ba94-e019bf0597b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:49:05.114844 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2b764-a285-4121-ba94-e019bf0597b3 service cinder] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:49:05.123593 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1433/2869] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:49:05 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:49:12.123736 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19442a-720f-4def-90bd-c4b3b560157b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:49:12.143387 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:49:12.187394 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19442a-720f-4def-90bd-c4b3b560157b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:49:12.192669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f713ed2-e92e-41e1-80f8-0bff52238521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:49:12.193530 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f713ed2-e92e-41e1-80f8-0bff52238521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:12.193837 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f713ed2-e92e-41e1-80f8-0bff52238521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:12.194110 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f713ed2-e92e-41e1-80f8-0bff52238521 admin admin] PATH_INFO: `/v3/users/27715ccd9ddd42ffbd8bb4b306ef4a0f` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:12.195267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f713ed2-e92e-41e1-80f8-0bff52238521 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27715ccd9ddd42ffbd8bb4b306ef4a0f)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:49:12.197197 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f713ed2-e92e-41e1-80f8-0bff52238521 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:49:12.242333 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0f713ed2-e92e-41e1-80f8-0bff52238521 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27715ccd9ddd42ffbd8bb4b306ef4a0f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:49:12.249019 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0f713ed2-e92e-41e1-80f8-0bff52238521 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27715ccd9ddd42ffbd8bb4b306ef4a0f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:49:12.249654 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0f713ed2-e92e-41e1-80f8-0bff52238521 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27715ccd9ddd42ffbd8bb4b306ef4a0f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:49:12.250249 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0f713ed2-e92e-41e1-80f8-0bff52238521 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27715ccd9ddd42ffbd8bb4b306ef4a0f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:49:12.262687 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0f713ed2-e92e-41e1-80f8-0bff52238521 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27715ccd9ddd42ffbd8bb4b306ef4a0f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:49:12.275405 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1437/2870] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:49:12 2022] DELETE /identity/v3/users/27715ccd9ddd42ffbd8bb4b306ef4a0f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:49:12.279967 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4f155-7e15-43b4-8a77-cae3cc6ea0ae None None] REQUEST_METHOD: `POST` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:12.280464 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4f155-7e15-43b4-8a77-cae3cc6ea0ae None None] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:12.280610 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4f155-7e15-43b4-8a77-cae3cc6ea0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:12.297050 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.auth.core [None req-7bc4f155-7e15-43b4-8a77-cae3cc6ea0ae None None] MFA Rules not processed for user `dc953d765b8048419427028bb5821273`. Rule list: `[]` (Enabled: `True`). {{(pid=80241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:49:12.298228 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-7bc4f155-7e15-43b4-8a77-cae3cc6ea0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:49:12.355800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1434/2871] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:49:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:49:12.365791 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f713ed2-e92e-41e1-80f8-0bff52238521 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:49:12.367190 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0f713ed2-e92e-41e1-80f8-0bff52238521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:49:12.424226 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f713ed2-e92e-41e1-80f8-0bff52238521 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:49:12.428441 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c85017-82c9-4b9e-b9ad-d5968a25a8d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:49:12.429079 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c85017-82c9-4b9e-b9ad-d5968a25a8d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:12.429359 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c85017-82c9-4b9e-b9ad-d5968a25a8d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:12.429631 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c85017-82c9-4b9e-b9ad-d5968a25a8d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:12.438538 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c85017-82c9-4b9e-b9ad-d5968a25a8d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:49:12.439995 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c85017-82c9-4b9e-b9ad-d5968a25a8d3 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:49:12.449312 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1438/2872] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:49:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:49:12.716671 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4f155-7e15-43b4-8a77-cae3cc6ea0ae None None] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:49:12.718376 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-7bc4f155-7e15-43b4-8a77-cae3cc6ea0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:49:12.731667 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4f155-7e15-43b4-8a77-cae3cc6ea0ae None None] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:49:12.737424 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c77dd-8830-46b9-a13b-5fa86836b23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:49:12.737903 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c77dd-8830-46b9-a13b-5fa86836b23e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:12.738965 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c77dd-8830-46b9-a13b-5fa86836b23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:12.739459 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c77dd-8830-46b9-a13b-5fa86836b23e admin admin] PATH_INFO: `/v3/projects/bb6fbb13fad9446196f3ea81383f160c` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:12.741571 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c77dd-8830-46b9-a13b-5fa86836b23e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb6fbb13fad9446196f3ea81383f160c)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:49:12.743439 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c77dd-8830-46b9-a13b-5fa86836b23e admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:49:12.790694 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3f1c77dd-8830-46b9-a13b-5fa86836b23e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb6fbb13fad9446196f3ea81383f160c', 'request_id': 'req-3f1c77dd-8830-46b9-a13b-5fa86836b23e'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:49:12.796345 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3f1c77dd-8830-46b9-a13b-5fa86836b23e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb6fbb13fad9446196f3ea81383f160c', 'request_id': 'req-3f1c77dd-8830-46b9-a13b-5fa86836b23e'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:49:12.803581 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3f1c77dd-8830-46b9-a13b-5fa86836b23e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb6fbb13fad9446196f3ea81383f160c', 'request_id': 'req-3f1c77dd-8830-46b9-a13b-5fa86836b23e'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:49:12.814445 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3f1c77dd-8830-46b9-a13b-5fa86836b23e admin admin] The token cache is being invalidate because project bb6fbb13fad9446196f3ea81383f160c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:49:12.814733 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3f1c77dd-8830-46b9-a13b-5fa86836b23e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:49:12.815363 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-3f1c77dd-8830-46b9-a13b-5fa86836b23e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:49:12.816981 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1435/2873] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:49:12 2022] DELETE /identity/v3/projects/bb6fbb13fad9446196f3ea81383f160c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:49:14.067743 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c85017-82c9-4b9e-b9ad-d5968a25a8d3 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:49:14.069192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-65c85017-82c9-4b9e-b9ad-d5968a25a8d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:49:14.080289 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c85017-82c9-4b9e-b9ad-d5968a25a8d3 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:49:14.119199 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:49:14.132230 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a30ff-88ce-4c70-a525-0efa85ffa439 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:49:14.132885 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a30ff-88ce-4c70-a525-0efa85ffa439 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:14.133172 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a30ff-88ce-4c70-a525-0efa85ffa439 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:14.133445 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a30ff-88ce-4c70-a525-0efa85ffa439 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:14.135244 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5c4a30ff-88ce-4c70-a525-0efa85ffa439 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:49:14.145148 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a30ff-88ce-4c70-a525-0efa85ffa439 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:49:14.146558 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a30ff-88ce-4c70-a525-0efa85ffa439 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:49:14.154515 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1439/2874] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:49:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:49:22.811507 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:49:24.150945 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=8 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:49:45.791354 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c77dd-8830-46b9-a13b-5fa86836b23e admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:49:45.792841 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-3f1c77dd-8830-46b9-a13b-5fa86836b23e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:49:45.805140 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c77dd-8830-46b9-a13b-5fa86836b23e admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:49:45.808686 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e46a27-70c4-4a8f-a69e-0e04e2986b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:49:45.809324 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e46a27-70c4-4a8f-a69e-0e04e2986b98 service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:45.809606 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e46a27-70c4-4a8f-a69e-0e04e2986b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:45.809882 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e46a27-70c4-4a8f-a69e-0e04e2986b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:45.811596 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-74e46a27-70c4-4a8f-a69e-0e04e2986b98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:49:45.881475 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e46a27-70c4-4a8f-a69e-0e04e2986b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:49:45.882821 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e46a27-70c4-4a8f-a69e-0e04e2986b98 service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:49:45.893996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1436/2875] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:49:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:49:47.067953 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a30ff-88ce-4c70-a525-0efa85ffa439 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:49:47.077028 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a30ff-88ce-4c70-a525-0efa85ffa439 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:49:47.080605 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d5ebb-63b6-4b7a-a5d4-8802a9dc73e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:49:47.081236 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d5ebb-63b6-4b7a-a5d4-8802a9dc73e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:47.081516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d5ebb-63b6-4b7a-a5d4-8802a9dc73e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:47.081787 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d5ebb-63b6-4b7a-a5d4-8802a9dc73e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:47.083696 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f26d5ebb-63b6-4b7a-a5d4-8802a9dc73e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:49:47.096912 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d5ebb-63b6-4b7a-a5d4-8802a9dc73e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:49:47.098853 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d5ebb-63b6-4b7a-a5d4-8802a9dc73e7 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:49:47.109830 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1440/2876] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:49:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:49:48.041189 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e46a27-70c4-4a8f-a69e-0e04e2986b98 service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:49:48.042564 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-74e46a27-70c4-4a8f-a69e-0e04e2986b98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:49:48.057850 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e46a27-70c4-4a8f-a69e-0e04e2986b98 service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:49:48.065371 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88cc1f-772f-4525-a00f-5d4633e23e6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:49:48.067025 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88cc1f-772f-4525-a00f-5d4633e23e6b service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:48.067646 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88cc1f-772f-4525-a00f-5d4633e23e6b service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:48.068172 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88cc1f-772f-4525-a00f-5d4633e23e6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:48.070139 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-cc88cc1f-772f-4525-a00f-5d4633e23e6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:49:48.086168 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88cc1f-772f-4525-a00f-5d4633e23e6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:49:48.087873 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88cc1f-772f-4525-a00f-5d4633e23e6b service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:49:48.099658 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1437/2877] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:49:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 10:49:49.407882 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d4a2b-33f4-4e8c-9630-d915c6c0839f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:49.408340 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d4a2b-33f4-4e8c-9630-d915c6c0839f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:49.408443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d4a2b-33f4-4e8c-9630-d915c6c0839f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:49.425459 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-687d4a2b-33f4-4e8c-9630-d915c6c0839f None None] MFA Rules not processed for user `8a7a5a9773d84145bbf611439d025dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:49:49.426745 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-687d4a2b-33f4-4e8c-9630-d915c6c0839f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:49:49.481399 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1441/2878] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:49:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:49:49.797372 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88cc1f-772f-4525-a00f-5d4633e23e6b service nova] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:49:49.807192 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88cc1f-772f-4525-a00f-5d4633e23e6b service nova] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:49:49.810878 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0deeca-f9f9-4160-9cf9-040b15f009cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:49:49.811505 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0deeca-f9f9-4160-9cf9-040b15f009cc service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:49.811810 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0deeca-f9f9-4160-9cf9-040b15f009cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:49.812108 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0deeca-f9f9-4160-9cf9-040b15f009cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:49.813861 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-ec0deeca-f9f9-4160-9cf9-040b15f009cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:49:49.824269 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0deeca-f9f9-4160-9cf9-040b15f009cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:49:49.825687 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0deeca-f9f9-4160-9cf9-040b15f009cc service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:49:49.833885 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1438/2879] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:49:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:49:51.281267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d4a2b-33f4-4e8c-9630-d915c6c0839f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:49:51.282643 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-687d4a2b-33f4-4e8c-9630-d915c6c0839f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:49:51.360773 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d4a2b-33f4-4e8c-9630-d915c6c0839f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:49:51.367016 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597834a1-36e0-4288-bd1c-deac07b4a6fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361be69f1c0a41c18afd206bccf56447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:49:51.367673 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597834a1-36e0-4288-bd1c-deac07b4a6fa service placement] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:49:51.367980 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597834a1-36e0-4288-bd1c-deac07b4a6fa service placement] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:49:51.368254 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597834a1-36e0-4288-bd1c-deac07b4a6fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:49:51.370035 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-597834a1-36e0-4288-bd1c-deac07b4a6fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:49:51.382081 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597834a1-36e0-4288-bd1c-deac07b4a6fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:49:51.383752 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597834a1-36e0-4288-bd1c-deac07b4a6fa service placement] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:49:51.400364 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1442/2880] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:49:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:49:59.830256 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=46 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:50:01.389804 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=63 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:50:07.589460 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0deeca-f9f9-4160-9cf9-040b15f009cc service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:50:07.604214 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0deeca-f9f9-4160-9cf9-040b15f009cc service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:50:07.609840 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e66474-6e48-445a-896b-2e9709bcfee4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:50:07.610488 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e66474-6e48-445a-896b-2e9709bcfee4 service nova] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:50:07.610775 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e66474-6e48-445a-896b-2e9709bcfee4 service nova] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:50:07.611054 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e66474-6e48-445a-896b-2e9709bcfee4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:50:07.612864 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-62e66474-6e48-445a-896b-2e9709bcfee4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:50:07.626020 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e66474-6e48-445a-896b-2e9709bcfee4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:50:07.627600 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e66474-6e48-445a-896b-2e9709bcfee4 service nova] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:50:07.640038 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1439/2881] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:50:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:50:11.802559 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597834a1-36e0-4288-bd1c-deac07b4a6fa service placement] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:50:11.845799 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597834a1-36e0-4288-bd1c-deac07b4a6fa service placement] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:50:11.853739 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1a766-91bf-4c99-9469-9abe8d199c0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8a40e4f1034a8e99d0eb505e448db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:50:11.854821 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1a766-91bf-4c99-9469-9abe8d199c0c service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:50:11.855334 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1a766-91bf-4c99-9469-9abe8d199c0c service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:50:11.855751 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1a766-91bf-4c99-9469-9abe8d199c0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:50:11.858235 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fbd1a766-91bf-4c99-9469-9abe8d199c0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:50:11.890618 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd1a766-91bf-4c99-9469-9abe8d199c0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:50:11.892169 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd1a766-91bf-4c99-9469-9abe8d199c0c service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:50:11.904993 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1443/2882] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:50:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:50:17.634673 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=6 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:50:21.900215 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=28 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:51:14.327497 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e66474-6e48-445a-896b-2e9709bcfee4 service nova] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:51:14.328970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-62e66474-6e48-445a-896b-2e9709bcfee4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:51:14.433484 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e66474-6e48-445a-896b-2e9709bcfee4 service nova] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:51:14.438760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d4aae-9f15-4d44-b2b8-2be247d10468 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361be69f1c0a41c18afd206bccf56447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:51:14.439402 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d4aae-9f15-4d44-b2b8-2be247d10468 service placement] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:51:14.439688 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d4aae-9f15-4d44-b2b8-2be247d10468 service placement] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:51:14.440022 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d4aae-9f15-4d44-b2b8-2be247d10468 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:51:14.441800 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a70d4aae-9f15-4d44-b2b8-2be247d10468 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:51:14.454422 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70d4aae-9f15-4d44-b2b8-2be247d10468 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:51:14.456032 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70d4aae-9f15-4d44-b2b8-2be247d10468 service placement] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:51:14.466688 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1440/2883] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:51:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:51:24.462435 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=37 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:51:45.670917 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b642c83-3c9d-4481-9c3e-7d7b40107c03 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:51:45.671449 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b642c83-3c9d-4481-9c3e-7d7b40107c03 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:51:45.671560 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b642c83-3c9d-4481-9c3e-7d7b40107c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:51:45.688669 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9b642c83-3c9d-4481-9c3e-7d7b40107c03 None None] MFA Rules not processed for user `8a7a5a9773d84145bbf611439d025dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 10:51:45.689866 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9b642c83-3c9d-4481-9c3e-7d7b40107c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:51:45.697340 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1444/2884] 10.222.0.42 () {58 vars in 957 bytes} [Mon Jun 6 10:51:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 10:51:55.683835 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:53:30.724446 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d4aae-9f15-4d44-b2b8-2be247d10468 service placement] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:53:30.725938 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-a70d4aae-9f15-4d44-b2b8-2be247d10468 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:53:30.760677 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d4aae-9f15-4d44-b2b8-2be247d10468 service placement] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:53:30.765994 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e6433-9954-4042-86f7-cfc3c09cbb2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:53:30.766627 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e6433-9954-4042-86f7-cfc3c09cbb2f service neutron] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:53:30.766906 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e6433-9954-4042-86f7-cfc3c09cbb2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:53:30.767177 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e6433-9954-4042-86f7-cfc3c09cbb2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:53:30.768950 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-eb1e6433-9954-4042-86f7-cfc3c09cbb2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:53:30.780421 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e6433-9954-4042-86f7-cfc3c09cbb2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:53:30.781823 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e6433-9954-4042-86f7-cfc3c09cbb2f service neutron] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:53:30.792905 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1441/2885] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:53:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:53:40.787028 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=15 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:53:43.990133 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b642c83-3c9d-4481-9c3e-7d7b40107c03 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:53:44.000028 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b642c83-3c9d-4481-9c3e-7d7b40107c03 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:53:44.003594 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baaf1f4-3717-40b3-9f21-4675d3368d7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '881069f9b40a485dac2dda684dd6cd3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:53:44.004267 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baaf1f4-3717-40b3-9f21-4675d3368d7c service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:53:44.004556 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baaf1f4-3717-40b3-9f21-4675d3368d7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:53:44.004834 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baaf1f4-3717-40b3-9f21-4675d3368d7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:53:44.006614 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5baaf1f4-3717-40b3-9f21-4675d3368d7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:53:44.090912 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baaf1f4-3717-40b3-9f21-4675d3368d7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:53:44.092388 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baaf1f4-3717-40b3-9f21-4675d3368d7c service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:53:44.111170 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1445/2886] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:53:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:53:47.210388 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e6433-9954-4042-86f7-cfc3c09cbb2f service neutron] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:53:47.225377 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e6433-9954-4042-86f7-cfc3c09cbb2f service neutron] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:53:47.232173 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1355af-3f58-4c9f-a54a-23c0b2ec7ac0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361be69f1c0a41c18afd206bccf56447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:53:47.233288 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1355af-3f58-4c9f-a54a-23c0b2ec7ac0 service placement] REQUEST_METHOD: `GET` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:53:47.233828 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1355af-3f58-4c9f-a54a-23c0b2ec7ac0 service placement] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:53:47.234118 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1355af-3f58-4c9f-a54a-23c0b2ec7ac0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:53:47.236114 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-5d1355af-3f58-4c9f-a54a-23c0b2ec7ac0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:53:47.249553 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1355af-3f58-4c9f-a54a-23c0b2ec7ac0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:53:47.251642 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1355af-3f58-4c9f-a54a-23c0b2ec7ac0 service placement] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:53:47.267921 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1442/2887] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:53:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:53:50.133709 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baaf1f4-3717-40b3-9f21-4675d3368d7c service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:53:50.136045 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5baaf1f4-3717-40b3-9f21-4675d3368d7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:53:50.150694 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baaf1f4-3717-40b3-9f21-4675d3368d7c service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:53:50.156557 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161940a-0087-4953-892e-ef614ee8bd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:53:50.157470 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161940a-0087-4953-892e-ef614ee8bd23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:53:50.157754 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161940a-0087-4953-892e-ef614ee8bd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:53:50.158073 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161940a-0087-4953-892e-ef614ee8bd23 admin admin] PATH_INFO: `/v3/users/8a7a5a9773d84145bbf611439d025dd8` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:53:50.159454 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c161940a-0087-4953-892e-ef614ee8bd23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a7a5a9773d84145bbf611439d025dd8)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:53:50.161516 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c161940a-0087-4953-892e-ef614ee8bd23 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:53:50.222258 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c161940a-0087-4953-892e-ef614ee8bd23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a7a5a9773d84145bbf611439d025dd8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:53:50.230409 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c161940a-0087-4953-892e-ef614ee8bd23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a7a5a9773d84145bbf611439d025dd8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:53:50.231076 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c161940a-0087-4953-892e-ef614ee8bd23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a7a5a9773d84145bbf611439d025dd8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:53:50.231678 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c161940a-0087-4953-892e-ef614ee8bd23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a7a5a9773d84145bbf611439d025dd8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:53:50.244400 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c161940a-0087-4953-892e-ef614ee8bd23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a7a5a9773d84145bbf611439d025dd8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:53:50.257044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1446/2888] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:53:50 2022] DELETE /identity/v3/users/8a7a5a9773d84145bbf611439d025dd8 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:53:50.431954 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1355af-3f58-4c9f-a54a-23c0b2ec7ac0 service placement] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:53:50.442633 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1355af-3f58-4c9f-a54a-23c0b2ec7ac0 service placement] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:53:50.499583 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c3029-9f80-41ea-96e7-a87eccb2d8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:53:50.500475 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c3029-9f80-41ea-96e7-a87eccb2d8fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:53:50.500760 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c3029-9f80-41ea-96e7-a87eccb2d8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:53:50.501035 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c3029-9f80-41ea-96e7-a87eccb2d8fb admin admin] PATH_INFO: `/v3/projects/6983c51eda664586ba62e206acfcc2d9` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:53:50.502202 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c3029-9f80-41ea-96e7-a87eccb2d8fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6983c51eda664586ba62e206acfcc2d9)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:53:50.504152 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c3029-9f80-41ea-96e7-a87eccb2d8fb admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:53:50.549727 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9f9c3029-9f80-41ea-96e7-a87eccb2d8fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6983c51eda664586ba62e206acfcc2d9', 'request_id': 'req-9f9c3029-9f80-41ea-96e7-a87eccb2d8fb'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:53:50.555271 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9f9c3029-9f80-41ea-96e7-a87eccb2d8fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6983c51eda664586ba62e206acfcc2d9', 'request_id': 'req-9f9c3029-9f80-41ea-96e7-a87eccb2d8fb'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:53:50.562886 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9f9c3029-9f80-41ea-96e7-a87eccb2d8fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6983c51eda664586ba62e206acfcc2d9', 'request_id': 'req-9f9c3029-9f80-41ea-96e7-a87eccb2d8fb'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:53:50.574431 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9f9c3029-9f80-41ea-96e7-a87eccb2d8fb admin admin] The token cache is being invalidate because project 6983c51eda664586ba62e206acfcc2d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:53:50.574740 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9f9c3029-9f80-41ea-96e7-a87eccb2d8fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:53:50.575370 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-9f9c3029-9f80-41ea-96e7-a87eccb2d8fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:53:50.576931 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1443/2889] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:53:50 2022] DELETE /identity/v3/projects/6983c51eda664586ba62e206acfcc2d9 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 10:53:50.580971 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161940a-0087-4953-892e-ef614ee8bd23 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:53:50.582443 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c161940a-0087-4953-892e-ef614ee8bd23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:53:50.596180 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161940a-0087-4953-892e-ef614ee8bd23 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:53:50.629160 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:53:50.654192 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396c886-deac-499e-8438-1875c55e9c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:53:50.655052 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396c886-deac-499e-8438-1875c55e9c1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:53:50.655335 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396c886-deac-499e-8438-1875c55e9c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:53:50.655611 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396c886-deac-499e-8438-1875c55e9c1f admin admin] PATH_INFO: `/v3/users/10b549e1e9fb4e34991fc90fc81d6f9e` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:53:50.656820 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396c886-deac-499e-8438-1875c55e9c1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=10b549e1e9fb4e34991fc90fc81d6f9e)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:53:50.658734 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396c886-deac-499e-8438-1875c55e9c1f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:53:50.703231 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3396c886-deac-499e-8438-1875c55e9c1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10b549e1e9fb4e34991fc90fc81d6f9e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:53:50.710119 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3396c886-deac-499e-8438-1875c55e9c1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10b549e1e9fb4e34991fc90fc81d6f9e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:53:50.710785 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3396c886-deac-499e-8438-1875c55e9c1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10b549e1e9fb4e34991fc90fc81d6f9e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:53:50.711396 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3396c886-deac-499e-8438-1875c55e9c1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10b549e1e9fb4e34991fc90fc81d6f9e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:53:50.723742 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3396c886-deac-499e-8438-1875c55e9c1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10b549e1e9fb4e34991fc90fc81d6f9e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:53:50.738002 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1447/2890] 10.222.0.42 () {58 vars in 1223 bytes} [Mon Jun 6 10:53:50 2022] DELETE /identity/v3/users/10b549e1e9fb4e34991fc90fc81d6f9e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:53:50.916911 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c3029-9f80-41ea-96e7-a87eccb2d8fb admin admin] Authenticating user token {{(pid=80241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:53:50.918449 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.fernet_utils [None req-9f9c3029-9f80-41ea-96e7-a87eccb2d8fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:53:50.932588 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c3029-9f80-41ea-96e7-a87eccb2d8fb admin admin] Validating token access rules against request {{(pid=80241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:53:50.996996 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9df4c7-6780-4430-9730-8306f8504d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc953d765b8048419427028bb5821273', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863195bccbf14e04a9a61a90f52ba9f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:53:50.997853 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9df4c7-6780-4430-9730-8306f8504d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:53:50.998158 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9df4c7-6780-4430-9730-8306f8504d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:53:50.998416 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9df4c7-6780-4430-9730-8306f8504d56 admin admin] PATH_INFO: `/v3/projects/9c36285723d74e1ea6ae42e57ad90b12` {{(pid=80241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:53:50.999580 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9df4c7-6780-4430-9730-8306f8504d56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c36285723d74e1ea6ae42e57ad90b12)` {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:53:51.001574 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9df4c7-6780-4430-9730-8306f8504d56 admin admin] RBAC: Authorization granted {{(pid=80241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:53:51.022508 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:53:51.049034 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-8d9df4c7-6780-4430-9730-8306f8504d56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c36285723d74e1ea6ae42e57ad90b12', 'request_id': 'req-8d9df4c7-6780-4430-9730-8306f8504d56'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:53:51.054395 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-8d9df4c7-6780-4430-9730-8306f8504d56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c36285723d74e1ea6ae42e57ad90b12', 'request_id': 'req-8d9df4c7-6780-4430-9730-8306f8504d56'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:53:51.061602 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-8d9df4c7-6780-4430-9730-8306f8504d56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c36285723d74e1ea6ae42e57ad90b12', 'request_id': 'req-8d9df4c7-6780-4430-9730-8306f8504d56'} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:53:51.073509 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-8d9df4c7-6780-4430-9730-8306f8504d56 admin admin] The token cache is being invalidate because project 9c36285723d74e1ea6ae42e57ad90b12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 10:53:51.073799 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-8d9df4c7-6780-4430-9730-8306f8504d56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:53:51.074414 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG keystone.notifications [None req-8d9df4c7-6780-4430-9730-8306f8504d56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 10:53:51.075970 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: [pid: 80241|app: 0|req: 1444/2891] 10.222.0.42 () {58 vars in 1232 bytes} [Mon Jun 6 10:53:50 2022] DELETE /identity/v3/projects/9c36285723d74e1ea6ae42e57ad90b12 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 10:54:00.734531 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:54:01.069834 dv-cinder-844822-1-iscsi devstack@keystone.service[80241]: DEBUG dbcounter [-] [80241] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 10:56:17.307548 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396c886-deac-499e-8438-1875c55e9c1f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 10:56:17.308971 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3396c886-deac-499e-8438-1875c55e9c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:56:17.377950 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396c886-deac-499e-8438-1875c55e9c1f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 10:56:17.438494 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b016574-0394-4046-8bfe-cb7f9b6f3fc6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361be69f1c0a41c18afd206bccf56447', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f082d7ed3694fd9998623f3d4a5fea9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 10:56:17.439147 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b016574-0394-4046-8bfe-cb7f9b6f3fc6 service placement] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 10:56:17.439432 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b016574-0394-4046-8bfe-cb7f9b6f3fc6 service placement] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 10:56:17.439709 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b016574-0394-4046-8bfe-cb7f9b6f3fc6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 10:56:17.441554 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5b016574-0394-4046-8bfe-cb7f9b6f3fc6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 10:56:17.452573 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b016574-0394-4046-8bfe-cb7f9b6f3fc6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 10:56:17.454143 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b016574-0394-4046-8bfe-cb7f9b6f3fc6 service placement] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 10:56:17.464044 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1448/2892] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jun 6 10:56:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 10:56:27.459621 dv-cinder-844822-1-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=58 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}